{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "Yu7u1mNfnxVP"
      },
      "source": [
        "**Copyright 2019 The Sonnet Authors. All Rights Reserved.**\n",
        "\n",
        "Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "you may not use this file except in compliance with the License.\n",
        "You may obtain a copy of the License at\n",
        "\n",
        "   http://www.apache.org/licenses/LICENSE-2.0\n",
        "\n",
        "Unless required by applicable law or agreed to in writing, software\n",
        "distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or  implied.\n",
        "See the License for the specific language governing permissions and\n",
        "limitations under the License.\n",
        "\n",
        "---"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "YlYRfLOzyuW9"
      },
      "source": [
        "## Generative Adversarial Networks (GANs)\n",
        "\n",
        "In this notebook we'll use Sonnet 2 and TensorFlow 2 to train a small image generator using the Generative Adversarial Nets (GAN) [1] framework. GANs consist of two modules:\n",
        "\n",
        "1.   A **generator**, which takes randomly sampled noise or latents as inputs and produces data (in this case, images) as output.\n",
        "2.   A **discriminator**, which provides the learning signal for the generator. Its inputs are real images and generated images, and it's trained to predict whether each input is real or generated.  The generator is trained to \"fool\" the discriminator into believing its outputs are real.\n",
        "\n",
        "Typically both the generator and discriminator are deep neural networks.\n",
        "\n",
        "For an extended tutorial on GANs, see Ian Goodfellow's [GAN Tutorial](https://arxiv.org/abs/1701.00160) [2].\n",
        "\n",
        "[1] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio. [Generative Adversarial Nets](https://papers.nips.cc/paper/5423-generative-adversarial-nets.pdf). *NeurIPS*, 2014.\n",
        "\n",
        "[2] I. Goodfellow. [NIPS 2016 Tutorial: Generative Adversarial Networks](https://arxiv.org/abs/1701.00160). *arXiv:1701.00160*, 2017."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "WAfR3cvnoGMB"
      },
      "source": [
        "# Preamble"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "4FqOAJb_jJR9"
      },
      "outputs": [],
      "source": [
        "import sys\n",
        "assert sys.version_info \u003e= (3, 6), \"Sonnet 2 requires Python \u003e=3.6\""
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "FZP3Wut53PTb"
      },
      "outputs": [],
      "source": [
        "!pip install dm-sonnet tqdm"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "ermIoeaTel6V"
      },
      "outputs": [],
      "source": [
        "import functools\n",
        "import time\n",
        "import numpy as np\n",
        "import sonnet as snt\n",
        "import tensorflow as tf\n",
        "import tensorflow_datasets as tfds\n",
        "import tqdm"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 51
        },
        "colab_type": "code",
        "id": "Rpp_houJEHr9",
        "outputId": "4edf64da-e887-4f06-90d7-e1e7e51e31fc"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "TensorFlow version: 2.0.0-rc1\n",
            "    Sonnet version: 2.0.0b0\n"
          ]
        }
      ],
      "source": [
        "print(\"TensorFlow version: {}\".format(tf.__version__))\n",
        "print(\"    Sonnet version: {}\".format(snt.__version__))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "ZAC0cJhzqE-a"
      },
      "source": [
        "Finally lets take a quick look at the GPUs we have available:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "colab_type": "code",
        "id": "6fOFkKYtqH8s",
        "outputId": "0f1bb7bb-0df2-492d-fd9e-67733a20cb28"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            " Tesla T4\n"
          ]
        }
      ],
      "source": [
        "!grep Model: /proc/driver/nvidia/gpus/*/information | awk '{$1=\"\";print$0}'"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "UYYmqvOKfNbk"
      },
      "source": [
        "# Dataset\n",
        "\n",
        "We need to get our dataset in a state where we can iterate over it easily. The TensorFlow Datasets package provides a simple API for this. It will download the dataset and prepare it for us to speedily process on a GPU. We can also add our own pre-processing functions to mutate the dataset before our model sees it:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "UkBRriaQEr4z"
      },
      "outputs": [],
      "source": [
        "def process_batch(images, labels):\n",
        "  images = tf.squeeze(images, axis=[-1])\n",
        "  images = tf.cast(images, dtype=tf.float32)\n",
        "  images /= 255.\n",
        "  images = tf.clip_by_value(images, 0., 1.)\n",
        "  return images, labels\n",
        "\n",
        "batch_size = 100\n",
        "def mnist(split, batch_size=batch_size):\n",
        "  dataset, ds_info = tfds.load('mnist:3.*.*', split=split, as_supervised=True,\n",
        "                               with_info=True)\n",
        "  dataset = dataset.map(process_batch)\n",
        "  dataset = dataset.batch(batch_size)\n",
        "  dataset = dataset.prefetch(tf.data.experimental.AUTOTUNE)\n",
        "  dataset = dataset.cache()\n",
        "  return dataset, ds_info\n",
        "\n",
        "mnist_train, mnist_train_info = mnist('train')\n",
        "mnist_test, mnist_test_info = mnist('test')\n",
        "\n",
        "mnist_shuffled = mnist_train.shuffle(10000).repeat()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "JfOCWVGEfgcq"
      },
      "source": [
        "MNIST contains `28x28` greyscale handwritten digits. Let's take a look at one:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 8,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 286
        },
        "colab_type": "code",
        "id": "I_yM0TVjFCZq",
        "outputId": "24c80259-d3e5-476d-e238-296506c02d62"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "\u003cmatplotlib.image.AxesImage at 0x7fb4d006c438\u003e"
            ]
          },
          "execution_count": 8,
          "metadata": {
            "tags": []
          },
          "output_type": "execute_result"
        },
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAADfxJREFUeJzt3X2MXXWdx/HPt53pVAqYVpY6W0Yo\nWMxWVlsy1oftEk2FAIsp/kOolBQlDmskSkJUUo1i1ii7i3UJGMIglcLyoBFIm1gfsJhFQCrDUwvO\nagu2sXXoAKPyoJRO+/WPOdUR5vzu7T3n3nNnvu9XcjP3nu95+ObCp+fe87v3/szdBSCeaVU3AKAa\nhB8IivADQRF+ICjCDwRF+IGgCD8QFOEHgiL8QFAdrTzYDOvymZrVykMCobyil/Wq77V61i0UfjM7\nXdJVkqZL+pa7X5Faf6Zm6d22rMghASRs9k11r9vwy34zmy7pm5LOkLRQ0gozW9jo/gC0VpH3/Esk\nbXf3p939VUm3S1peTlsAmq1I+OdJ+u24x7uyZX/HzPrMbMDMBvZpb4HDAShT06/2u3u/u/e6e2+n\nupp9OAB1KhL+3ZJ6xj0+JlsGYBIoEv6HJC0ws/lmNkPSuZI2lNMWgGZreKjP3UfN7GJJP9LYUN9a\nd3+ytM4ANFWhcX533yhpY0m9AGghPt4LBEX4gaAIPxAU4QeCIvxAUIQfCKql3+fH5DNtUfqLml1X\nPZ+s/3m0M7+4bFcjLaEknPmBoAg/EBThB4Ii/EBQhB8IivADQTHUF9z0I49M1uddtzNZv77n/mT9\n+O9dlFtbIIb6qsSZHwiK8ANBEX4gKMIPBEX4gaAIPxAU4QeCYpw/uO3XzU/WN/asS9b7//iPyfr8\n9aOH3BNagzM/EBThB4Ii/EBQhB8IivADQRF+ICjCDwRVaJzfzHZIelHSfkmj7t5bRlMoz+8+875k\nffCUa2rsIX1+uO5/lifrR236eY39oyplfMjnA+7+XAn7AdBCvOwHgioafpf0YzN72Mz6ymgIQGsU\nfdm/1N13m9nRku42s/9393vHr5D9o9AnSTN1WMHDAShLoTO/u+/O/g5LukvSkgnW6Xf3Xnfv7VRX\nkcMBKFHD4TezWWZ2xMH7kk6T9ERZjQForiIv++dKusvMDu7nVnf/YSldAWi6hsPv7k9LemeJvaBB\n02bOzK1d8rE7k9tOt/SLvy89+/Zk/eibH0/WDySrqBJDfUBQhB8IivADQRF+ICjCDwRF+IGg+Onu\nKWDPBYtzaxe+8cFC+/7BmlOS9dl/4iu7kxVnfiAowg8ERfiBoAg/EBThB4Ii/EBQhB8IinH+SaBj\nXnoa7O+v/u9E9fDktieu+0SyPv+mYp8TQPvizA8ERfiBoAg/EBThB4Ii/EBQhB8IivADQTHOPwkM\nfq4nWe/uyB/Lf27/y8lt569P1+WermPS4swPBEX4gaAIPxAU4QeCIvxAUIQfCIrwA0HVHOc3s7WS\nzpI07O4nZcvmSPqOpOMk7ZB0jrv/vnltTm0db56brN/+oWtq7KEzt3L1yJL0pg9uqbFvTFX1nPlv\nlHT6a5ZdJmmTuy+QtCl7DGASqRl+d79X0shrFi+XtC67v07S2SX3BaDJGn3PP9fdh7L7z0hKv24F\n0HYKX/Bzd5eU+wFwM+szswEzG9invUUPB6AkjYZ/j5l1S1L2dzhvRXfvd/ded+/tVFeDhwNQtkbD\nv0HSquz+Kknry2kHQKvUDL+Z3Sbp55LeZma7zOxCSVdIOtXMtkn6YPYYwCRSc5zf3VfklJaV3Etc\nh70hWV7SlT+OX8sDn0qP80/Tow3vu9k6eo5J1g/MOSJdf3ywzHamHD7hBwRF+IGgCD8QFOEHgiL8\nQFCEHwiKn+5uA785Lz0Fdy17fV9ubdqfRwvtuyjryv9U587/PTG57VWLb0/WF3Smv0V+/qWX5tZm\nfW9zctsIOPMDQRF+ICjCDwRF+IGgCD8QFOEHgiL8QFCM87dAR/ebk/WrL7iu0P6/OPyu/OIvthba\ndy2pcXxJev6OY3NrgyffXPDo+VOTS9KX//NbubU195+a3HZ06JmGOppMOPMDQRF+ICjCDwRF+IGg\nCD8QFOEHgiL8QFCM87fAy4t7kvVlb9hfaP9PvXRUovpcoX3XPPaXT07Wt518bcP7Hnz1T8n6P804\nLFlPPa//sSj936SLcX4AUxXhB4Ii/EBQhB8IivADQRF+ICjCDwRVc5zfzNZKOkvSsLuflC27XNLH\nJT2brbba3Tc2q0mkDf4g//fvjyk4zv+br703Wb//vCtr7GFWbuWbf0iPtd945VnJ+kNfSX+GYL8f\nyK3ZAU9uG0E9Z/4bJZ0+wfJvuPui7EbwgUmmZvjd/V5JIy3oBUALFXnPf7GZbTGztWY2u7SOALRE\no+G/VtIJkhZJGpL09bwVzazPzAbMbGCf9jZ4OABlayj87r7H3fe7+wFJ10takli339173b23U+kf\newTQOg2F38y6xz38sKQnymkHQKvUM9R3m6T3SzrKzHZJ+pKk95vZIkkuaYeki5rYI4AmqBl+d18x\nweIbmtDLlNU1kr7WMTT6UrLe3ZH+ffojlg4fck8HdczP/119SbpvZXoc/+jp+eP4kvSF4X/OrT36\nb+lx/pErXknWazl/x7Lc2owfDRTa91TAJ/yAoAg/EBThB4Ii/EBQhB8IivADQfHT3a3w4JZk+ern\n35esf3Vuevt73nFrbu2D534que2cf9+ZrNcayqvl1sdyP/ypzjX7kts+9a/fLnTsP/Qdnaj+vtC+\npwLO/EBQhB8IivADQRF+ICjCDwRF+IGgCD8QlLm37ieMj7Q5/m7L/5plVAeWLkrW7/7uja1pZJJZ\n+MDKZP3YldtzawdeKfZ14Xa12TfpBR+xetblzA8ERfiBoAg/EBThB4Ii/EBQhB8IivADQfF9/jbQ\n8fhTyfpbb/lEsj74kWtya502vaGeWmFXjZ8sP63/s8l6z1ceSNbzJ+iGxJkfCIvwA0ERfiAowg8E\nRfiBoAg/EBThB4Kq+X1+M+uRdJOkuZJcUr+7X2VmcyR9R9JxknZIOsfdkz+Gzvf5m+P5j783t/ah\ni/8vuW3f7F8k67WmBy/ihHs+mqy/deWjTTv2VFX29/lHJV3q7gslvUfSJ81soaTLJG1y9wWSNmWP\nAUwSNcPv7kPu/kh2/0VJg5LmSVouaV222jpJZzerSQDlO6T3/GZ2nKTFkjZLmuvuQ1npGY29LQAw\nSdQdfjM7XNIdki5x9xfG13zswsGEFw/MrM/MBsxsYJ/2FmoWQHnqCr+ZdWos+Le4+53Z4j1m1p3V\nuyUNT7Stu/e7e6+793aqq4yeAZSgZvjNzCTdIGnQ3deMK22QtCq7v0rS+vLbA9As9Qz1LZX0M0lb\n9bdvSa7W2Pv+70p6i6SdGhvqG0nti6G+9vPHle9J1s9f/f1kfdWR25L1d9xxSW7txM+mh/J8L28T\nD9WhDPXV/D6/u98nKW9nJBmYpPiEHxAU4QeCIvxAUIQfCIrwA0ERfiAopugGphCm6AZQE+EHgiL8\nQFCEHwiK8ANBEX4gKMIPBEX4gaAIPxAU4QeCIvxAUIQfCIrwA0ERfiAowg8ERfiBoAg/EBThB4Ii\n/EBQhB8IivADQRF+ICjCDwRVM/xm1mNmPzWzX5rZk2b26Wz55Wa228wey25nNr9dAGXpqGOdUUmX\nuvsjZnaEpIfN7O6s9g13v7J57QFolprhd/chSUPZ/RfNbFDSvGY3BqC5Duk9v5kdJ2mxpM3ZoovN\nbIuZrTWz2Tnb9JnZgJkN7NPeQs0CKE/d4TezwyXdIekSd39B0rWSTpC0SGOvDL4+0Xbu3u/uve7e\n26muEloGUIa6wm9mnRoL/i3ufqckufsed9/v7gckXS9pSfPaBFC2eq72m6QbJA26+5pxy7vHrfZh\nSU+U3x6AZqnnav+/SDpf0lYzeyxbtlrSCjNbJMkl7ZB0UVM6BNAU9Vztv0/SRPN9byy/HQCtwif8\ngKAIPxAU4QeCIvxAUIQfCIrwA0ERfiAowg8ERfiBoAg/EBThB4Ii/EBQhB8IivADQZm7t+5gZs9K\n2jlu0VGSnmtZA4emXXtr174kemtUmb0d6+7/UM+KLQ3/6w5uNuDuvZU1kNCuvbVrXxK9Naqq3njZ\nDwRF+IGgqg5/f8XHT2nX3tq1L4neGlVJb5W+5wdQnarP/AAqUkn4zex0M/uVmW03s8uq6CGPme0w\ns63ZzMMDFfey1syGzeyJccvmmNndZrYt+zvhNGkV9dYWMzcnZpau9LlrtxmvW/6y38ymS/q1pFMl\n7ZL0kKQV7v7LljaSw8x2SOp198rHhM3sFEkvSbrJ3U/Klv2XpBF3vyL7h3O2u3+uTXq7XNJLVc/c\nnE0o0z1+ZmlJZ0u6QBU+d4m+zlEFz1sVZ/4lkra7+9Pu/qqk2yUtr6CPtufu90oaec3i5ZLWZffX\naex/npbL6a0tuPuQuz+S3X9R0sGZpSt97hJ9VaKK8M+T9Ntxj3epvab8dkk/NrOHzayv6mYmMDeb\nNl2SnpE0t8pmJlBz5uZWes3M0m3z3DUy43XZuOD3ekvd/WRJZ0j6ZPbyti352Hu2dhquqWvm5laZ\nYGbpv6ryuWt0xuuyVRH+3ZJ6xj0+JlvWFtx9d/Z3WNJdar/Zh/ccnCQ1+ztccT9/1U4zN080s7Ta\n4Llrpxmvqwj/Q5IWmNl8M5sh6VxJGyro43XMbFZ2IUZmNkvSaWq/2Yc3SFqV3V8laX2Fvfyddpm5\nOW9maVX83LXdjNfu3vKbpDM1dsX/KUmfr6KHnL6Ol/R4dnuy6t4k3aaxl4H7NHZt5EJJb5K0SdI2\nST+RNKeNertZ0lZJWzQWtO6KeluqsZf0WyQ9lt3OrPq5S/RVyfPGJ/yAoLjgBwRF+IGgCD8QFOEH\ngiL8QFCEHwiK8ANBEX4gqL8A74xLCC0psmEAAAAASUVORK5CYII=\n",
            "text/plain": [
              "\u003cFigure size 432x288 with 1 Axes\u003e"
            ]
          },
          "metadata": {
            "tags": []
          },
          "output_type": "display_data"
        }
      ],
      "source": [
        "import matplotlib.pyplot as plt\n",
        "\n",
        "images, _ = next(iter(mnist_test))\n",
        "plt.imshow(images[0])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "d7bsizs5gK3K"
      },
      "source": [
        "# Sonnet\n",
        "\n",
        "The next step is to define a model. In Sonnet everything that contains TensorFlow variables (`tf.Variable`) extends `snt.Module`, this includes low level neural network components (e.g. `snt.Linear`,  `snt.Conv2D`), larger nets containing subcomponents (e.g. `snt.nets.MLP`), optimizers (e.g. `snt.optimizers.Adam`) and whatever else you can think of.\n",
        "\n",
        "Modules provide a simple abstraction for storing parameters (and `Variable`s used for other purposes, like for storing moving averages in `BatchNorm`).\n",
        "\n",
        "To find all the parameters for a given module, simply do: `module.variables`. This will return a `tuple` of all the parameters that exist for this module, or any module it references:"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "GrN37pi1o4HT"
      },
      "source": [
        "## Building the model"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "c6XoN56S2lSW"
      },
      "source": [
        "In Sonnet you build neural networks out of `snt.Module`s. In this simple example we'll build multi-layer perceptron (MLP) based generators and discriminators.\n",
        "\n",
        "We'll make use of \"Spectral Normalization\" [1] in both modules, which serves to regularize them.\n",
        "\n",
        "[1] T. Miyato, T. Kataoka, M. Koyama, and Y. Yoshida. [Spectral Normalization for Generative Adversarial Networks](https://arxiv.org/abs/1802.05957). *arXiv:1802.05957*, 2018."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "hgjyB9yhFclD"
      },
      "outputs": [],
      "source": [
        "class SpectralNormalizer(snt.Module):\n",
        "\n",
        "  def __init__(self, epsilon=1e-12, name=None):\n",
        "    super().__init__(name=name)\n",
        "    self.l2_normalize = functools.partial(tf.math.l2_normalize, epsilon=epsilon)\n",
        "\n",
        "  @snt.once\n",
        "  def _initialize(self, weights):\n",
        "    init = self.l2_normalize(snt.initializers.TruncatedNormal()(\n",
        "        shape=[1, weights.shape[-1]], dtype=weights.dtype))\n",
        "    # 'u' tracks our estimate of the first spectral vector for the given weight.\n",
        "    self.u = tf.Variable(init, name='u', trainable=False)\n",
        "\n",
        "  def __call__(self, weights, is_training=True):\n",
        "    self._initialize(weights)\n",
        "    if is_training:\n",
        "      # Do a power iteration and update u and weights.\n",
        "      weights_matrix = tf.reshape(weights, [-1, weights.shape[-1]])\n",
        "      v = self.l2_normalize(self.u @ tf.transpose(weights_matrix))\n",
        "      v_w = v @ weights_matrix\n",
        "      u = self.l2_normalize(v_w)\n",
        "      sigma = tf.stop_gradient(tf.reshape(v_w @ tf.transpose(u), []))\n",
        "      self.u.assign(u)\n",
        "      weights.assign(weights / sigma)\n",
        "    return weights\n",
        "\n",
        "\n",
        "class SpectrallyNormedLinear(snt.Linear):\n",
        "\n",
        "  def __init__(self, *args, **kwargs):\n",
        "    super().__init__(*args, **kwargs)\n",
        "    self.spectral_normalizer = SpectralNormalizer()\n",
        "\n",
        "  def __call__(self, inputs, is_training=True):\n",
        "    self._initialize(inputs)\n",
        "\n",
        "    normed_w = self.spectral_normalizer(self.w, is_training=is_training)\n",
        "    outputs = tf.matmul(inputs, normed_w)\n",
        "    if self.with_bias:\n",
        "      outputs = tf.add(outputs, self.b)\n",
        "    return outputs\n",
        "\n",
        "\n",
        "class SimpleBlock(snt.Module):\n",
        "\n",
        "  def __init__(self, embed_dim, with_batch_norm=False, name=None):\n",
        "    super().__init__(name=name)\n",
        "    self.embed_dim = embed_dim\n",
        "    self.hidden = SpectrallyNormedLinear(self.embed_dim)\n",
        "    if with_batch_norm:\n",
        "      self.bn = snt.BatchNorm(create_scale=True, create_offset=True)\n",
        "    else:\n",
        "      self.bn = None\n",
        "\n",
        "  def __call__(self, inputs, is_training=True):\n",
        "    output = self.hidden(inputs, is_training=is_training)\n",
        "    if self.bn:\n",
        "      output = self.bn(output, is_training=is_training)\n",
        "    output = tf.nn.relu(output)\n",
        "    return output\n",
        "\n",
        "\n",
        "class Generator(snt.Module):\n",
        "\n",
        "  def __init__(self, output_shape, num_layers=1, embed_dim=1024, name=None):\n",
        "    super().__init__(name=name)\n",
        "    self.layers = [\n",
        "        SimpleBlock(embed_dim, with_batch_norm=True, name='block_'+str(index))\n",
        "        for index in range(num_layers)\n",
        "    ]\n",
        "    self.output_shape = tuple(output_shape)\n",
        "    output_size = np.prod(self.output_shape, dtype=int)\n",
        "    self.outputs = snt.Linear(output_size, name='outputs')\n",
        "\n",
        "  def __call__(self, inputs, is_training=True):\n",
        "    inputs = tf.convert_to_tensor(inputs)\n",
        "    output = snt.Flatten()(inputs)\n",
        "    for layer in self.layers:\n",
        "      output = layer(output, is_training=is_training)\n",
        "    output = self.outputs(output)\n",
        "    output = tf.reshape(output, [-1] + list(self.output_shape))\n",
        "    output = tf.sigmoid(output)\n",
        "    return output\n",
        "\n",
        "\n",
        "class Discriminator(snt.Module):\n",
        "\n",
        "  def __init__(self, num_layers=1, embed_dim=1024, name=None):\n",
        "    super().__init__(name=name)\n",
        "    self.layers = [\n",
        "        SimpleBlock(embed_dim, with_batch_norm=False, name='block_'+str(index))\n",
        "        for index in range(num_layers)\n",
        "    ]\n",
        "    self.outputs = SpectrallyNormedLinear(1, name='outputs')\n",
        "\n",
        "  def __call__(self, inputs, is_training=True):\n",
        "    inputs = tf.convert_to_tensor(inputs)\n",
        "    output = snt.Flatten()(inputs)\n",
        "    for layer in self.layers:\n",
        "      output = layer(output, is_training=is_training)\n",
        "    output = self.outputs(output)\n",
        "    return tf.reshape(output, [-1])\n",
        "\n",
        "\n",
        "class LittleGAN(snt.Module):\n",
        "\n",
        "  def __init__(self, num_layers=2, embed_dim=1024, name=None):\n",
        "    super().__init__(name=name)\n",
        "    self.generator = Generator(\n",
        "        [28, 28], num_layers=num_layers, embed_dim=embed_dim)\n",
        "    self.discriminator = Discriminator(\n",
        "        num_layers=num_layers, embed_dim=embed_dim)\n",
        "\n",
        "  def generate(self, noise, is_training=True):\n",
        "    return self.generator(noise, is_training=is_training)\n",
        "\n",
        "  def discriminate(self, images):\n",
        "    return self.discriminator(images)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "0i03px8y8gf7"
      },
      "source": [
        "Now we'll create an instance of our class whose weights will be randomly initialized. We'll train this MLP such that it learns to recognize digits in the MNIST dataset."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 10,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "colab_type": "code",
        "id": "XqL8oIMqGAnU",
        "outputId": "6471fe0d-0db3-4da0-c4ce-cdf672da67d9"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "LittleGAN(num_layers=2)"
            ]
          },
          "execution_count": 10,
          "metadata": {
            "tags": []
          },
          "output_type": "execute_result"
        }
      ],
      "source": [
        "gan = LittleGAN(num_layers=2)\n",
        "gan"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "snzkUUh9oXPy"
      },
      "source": [
        "## Using the model"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "On8wI6VwpDPm"
      },
      "source": [
        "Let's feed some random noise through the generator and see what it generates. Since the model is randomly initialized and not trained yet, the images it produces should look like noise.\n",
        "\n",
        "Below, the top row of images are real MNIST digits; the bottom row are the outputs of our randomly initialized generator."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 11,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 160
        },
        "colab_type": "code",
        "id": "4T-qmIc0GHfP",
        "outputId": "35babc71-6172-4756-f3fb-bb2f6f5cf363"
      },
      "outputs": [
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAABFQAAACPCAYAAADUS4+vAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzsnWd4VNXWgN8zJb0nkIQUeiiKCAhi\nBUQUCypWsCsWrGCv31Wv3nvtimIDRbGjiIKCYgOkShGQ3lsghBTS68yc78c6E4gIEkibyXqfJ8/M\nnDnnzM5eZ+2y1tprG6ZpoiiKoiiKoiiKoiiKohw+toYugKIoiqIoiqIoiqIoiq+hBhVFURRFURRF\nURRFUZQaogYVRVEURVEURVEURVGUGqIGFUVRFEVRFEVRFEVRlBqiBhVFURRFURRFURRFUZQaogYV\nRVEURVEURVEURVGUGqIGFUVRFEVRFEVRFEVRlBpyVAYVwzAGGoaxzjCMjYZhPFxbhVLqF5Wj76My\n9A9Ujr6PytA/UDn6PipD/0Dl6PuoDP0DlePBMUzTPLILDcMOrAcGAOnAImCoaZqra694Sl2jcvR9\nVIb+gcrR91EZ+gcqR99HZegfqBx9H5Whf6ByPDSOo7i2F7DRNM3NAIZhfA5cCBy0YgOMQDOI0KP4\nyYanjGIqzHKjoctRi9RIjv4gQ4BC9mabptmsoctRS6gu+geqi76P6qJ/oLro+6gu+geqi76P6qJ/\noLp4CI7GoJIE7Njvczpw4l9PMgzjFuAWgCBCONHofxQ/2fD8bv7S0EWobf5Rjv4mQ4CfzYnbGroM\ntYjqon+guuj7qC76B6qLvo/qon+guuj7qC76B6qLh6DOk9KapjnGNM0TTNM8wUlgXf+cUgeoDP0D\nlaPvozL0D1SOvo/K0D9QOfo+KkP/QOXo+zRlGR5NhMpOIGW/z8nWsQbFdnxnAAJH5QBQ6nJC//SG\nLFJjp1HKUakRKkP/QOXo+6gM/YMGl+MFq2UMc0fUjmrH234xHIAOT67BnZdfn0XyNRpchkqtoHL0\nfVSG/oHK8RAcjUFlEdDeMIzWSIUOAa6slVIdAfaICACS3pHInLEpcwFoM/FW2qMGlUPQqOSoHBEq\nQ/9A5ej7qAz9gwaX4xufDQJg+PDR1Y5vuvxtANIvLmLIvfcBEDrx9/osmq/Q4DJUagWVo++jMvQP\nfEKO5ef0BODh1z8E4Ovc7gAsGXM8se/Or7PfPWKDimmaLsMw7gSmA3ZgnGmaq2qtZEq9oHL0fVSG\n/oHK0fdRGfoHKkffR2XoH6gcfR+VoX+gcjw0RxOhgmma04BptVSWo2LjO60BmJYyHoAx+S0AaD3Z\n1WBl8hUajRx7dQGgxWtbAXg/dTYAnxdG887dlwAQMH1xgxStsVPXMiy9qBcA2V2kySjvUArA2n7v\nVp3jNOwAVJpuADr+ehMAgeuDAYhb4SL4m4V1VUS/oNHoonLEqAz9g4aW48NXfXHI7xPtIYx/6SUA\nbs4bAYDz5yV1Xi5foqFlqNQOKkffR2XoHzR2OaY/cjITb30RgHZOyeEywynzlWaL8/HU4W/XeVJa\nRVEURVEURVEURVEUf+OoIlQaA7seOBmANad71xmLjeidVy8EIO6XulsvpdQO3vVur731OgDHOAMA\ncJvy/WVhORS/+h0AX10g8nZv2FzPpWx6OFKSASgZZ2d0+9cA6OC0Vztnf2tvpek9JkdXnzFGDpwh\nLxsrXdxx+1AAQodJ5Jhrh+Y3UhRF+SvPfH0ZAE+3FO9a5AyJ9Lvs7p8BuD9mHa0cIQDs7i2euJSf\n67uUyv5M37UM2BeluT+dZw0DIGRRSLXjzf8oxTZrad0XTjksvPkYPaVlAJiVFf94jS0khE3vtQdg\n1enjADj14TsBiPpI5yCKUtdsefYkAOZd9QK51sSk2+t3AZD65koAPAWr67QMGqGiKIqiKIqiKIqi\nKIpSQ3w6QsUWFMTIGycBYDfENvRE1jEANP9oOUCdrpdSjh57XCz/fmMsAM1sErVw/MIbAUh4Wbxu\nWy4MZN3QNwEom7wIgKmntAPAvXdvvZa3KVHYQ/IQvZ/2MjvdYfI+PwmAd96UCLDg7H0aZhryaliR\nKgE37gZgSLLIrEvQDqYf8yUAj0+WnCyrrFxH7n676urfUJQmQeEVvQEIn7CAjPskks8VJN+lPie5\ni0yX5hTzFVo/8vee7VkTEgHImBHJSwki129vfh6AIbseACBmnHrF6xOvd7TSXGK9Hhihsvx0idh0\n9qmea+zr4kS+z+lS7dx5izsA0OGRlXiKi+um0MrfsmXksQAk95Htym39dxzqdACyhnZlxekSxesd\nET32L8nn+MZHabVfSEVp4hjWSob1Y6Xt3DDgDQCey+nOjNulPU6aMw+AA1vjusGnDSqZ13djWOSC\nase+f/l0AKJLdEDhC6x9qh2nBP4EwAn/fRCAFm/Mq3ZO2zkGXVpdC8DSk94HYPRdsqVk6r+rn6vU\nHt4EskMjHyByk4Sd2+ZISHNzDqPeP5eXr2kGwFenncWT498D4Jl4KzltvLyccvPdxL5r6bJp1kLp\nFcX/MBwOjI5iTC5LEiPnMc+sAODqWBlQfHz/yXyeIEnZwmxilL5vsBhbCl3hAOy5LFKX2/ko7oIC\nADacn8BPs2UZ0AB54fhb/wRg+7gGKVqTJWrtkV87ODSDwaEZ1Y45W/4CQP9fbiN4siZyrw9yhskk\nbPKwFwBo6ZAJ2wX0PKL7JTnyALCntcW9flMtlFBRFC+ZE9sAsO4EMVTPLnMCMPfCjti2LGuQMumS\nH0VRFEVRFEVRFEVRlBrikxEqjiRZJjD10RcA8dKljb8NgNYfLjjYZUojpOOTm+m1/A4AEj4Wq+IB\ny7RMk5RLJanQA4sllP1/13wIwJh3TsaduadeytpUiR5fO9FettlL+Xeb7gDYZ4gOf502GYD5T46m\nR4QkkEp8SaOOjpa914u3LXy7JNRz/LpvO9Wy82W5VWVodXt65Jo8PH8ehatVqXPscbF8M/2TQ57T\no8U8ILDasZcSq/eLj0/pwZTJ0pamPqn65ou4MnZz9xeyPHbNdRKd9H+J0wG4se/d2Gf+0WBla2pE\nfyB95IULhx7wXV6XGABS714PwClREq0wLHLDP94397pikibXVimVv8MeK/IZdPcsYF9kSk1oviCX\nWaWSbLhPcAkAOe5QAMwMHZ/WNbk3nkRO33IAnj5RFObZ1WcDULYu8oDzk36Tpa87Tz9wCpw0y/qu\nj3x3Up9VAOy+vw3G3IaJfFD2UXClRNvO7vEqAOnWMuYX+lwAgCt9W8MUDI1QURRFURRFURRFURRF\nqTE+GaGy5qEUABIdYWS7JWFX68lW4i7Nv+BTuLOyiBuTBRxeAuEZn8t61lfu+R2AdxJiQSNUfAbv\nVsydI3ce8F3syn/enlA5NFetlbwYV4RLgjy31R6muyurzikzJVrBjnyXZiX3KjErKDdFC72W9vfy\njgfgq+3yWv5TM5J+ypF7r1pXV/+G8hcGr5Y20mbsa+v2emRbz9dyxGPjsbJC2wyz2nuATsGS9Pny\nMLn+meZLePQm8ar3ct4LQKvHNO+YrxG8x6j2eVl5cwACsorrLRGfsg/36vUHHAu3durcO0Fe373n\nPACG3ffqP94vZnxorZVN+Xt2XdkRgEfjfrKOSO/X8QuJnG7HP0e9u1et4zurr+wXLOdXIsmHPYWF\ntVlc5W/I7uVmff+x1Y5d3usjedPrwPOLrpJoFm+Osf3Jv0r61UhbULXjX4xbxYcdUmqhtMqRYju2\nIw899TEATkP0a+hjIwGITG/41SkaoaIoiqIoiqIoiqIoilJDfCpCxZEgW4J8Pmi0dcTJ67mW+XHB\nnw1TKKVeidimfjdfwRYkFn6zU1sAdvaPZOCV4gWv2uXH4vT77yB6vrjyVMJHzoe3yTrS0anieQnO\nldoMzigBt0QrmEtlTTA2sfCXndej6vodA8TjbYbLutT7T/wRgOnHyRaQYccHknmv7Ph0wf9ki9Zm\nb2lkQ10zPEoiuipNN7eny052C77sCkCLF/85B8ribgMBeOryCABWXTuaEEMikypiVeN8leSvtgPw\n0fAEAK4Jl63qH7osltRVDVYsZT8crVsC0OLzbACmp7wJQKVpP+g1XedfB0DqzhI05rpuuf32bwDw\nWDHS9+w6DYC0f62yjh8eHtNW7T5uU/3V9cX9p39fo/Oj7ZLv5u+2N/9rZIqXKLtuX97QrL09gvNC\n8gE45rebAGj9ccNHpnhRjVcURVEURVEURVEURakhPhWhQkgwAL0CnVWH5t0tESo2lv7j5d78DZ6Y\ncHldvqa2S6jUMZm9xAa4tlLWQBrFZQ1ZnKZL7+MAWH/LfhnxrXwNWPkbQiIlkuGP3h8AYMNW5b35\nvVx0ePi42wFI+WyeRqbUAvYZsrNH9F+O/62X0yM1HvTtvmih9t9WP2UKsQC89sx9AKy8YTTxdmmH\ni2WjJpodVYmVw2HIljMA2P56GlE/y+4gLbIPf3ceb1RSmxXS5XdqOYw1fd6r5VIq9Y1rh+RMenuz\nRC1d0/ULAFr32artaQNiWHmptj12ApcPlt1jHoqVMao3MsXrHV9TCWsrEgF48uvLAWjzsET9aXRK\n3XNDxA5gXyTKosxUAGIKD8yHozQucm+Q3QyvCH8RqB5Zcl+G5Bb7I/vAvCdh/ye5iTJOlblg+Nm7\nDzjnubSJAPQKVC1saAqGWjv7nP8iIONP1x55NXp2AcCeXSDHtzTcLj8+ZVDZclWLap/LzUpspa5D\nXmMEBrLt4zQARnX7HID2zr0AXHPffYRO/L0OSqrUFS27Sej7uJxTAHBv3NKQxWk6GGIkKZjWBoDf\njnv/gFO8SaIODKMUI9gLOZ355LP+ACT/TyaDKeiWrb7A/tHpXqNY86WHGwytHC17T8kFIJwFRzVR\nNoJlEKLGFD/BWrYX5JBxkN2QtrbU5aTmm78qtYUtrTUAS246MPHsM9nijPjkZ1la0mK2h+BvxKjd\nBl0+WZ/sevBkYElDF0M5QioiZFy6/zKdAasuASD0WnHohe7efMB1XhNJwiLrzSsH3nvUnAEAfNL6\nx9oprHLEzHlRlkl6LGMKwLpL35A3l8pLvpWk/8P8Lry3TgxtKUPE+WRW1s+GF7rkR1EURVEURVEU\nRVEUpYb4RISKI1ESrr1+/TvVjv9rT09YuOJvrzECJSljzlctWdP9o798GwbAU8+9y8tzxQrpyjgw\n5EtpPJgnSQLGyR3lGej223AA2rCswcrUFCn+WRJDp3cW638Lx75t5yots7/nIGncfsrsSMLC8rot\noFInuBL3WfjfzZcopZBJGt2nKA3JjkdOBGDFMZKo/8MC2TY5+DabLvlppMx69GQA2k5tPMkUmyqV\nYeZ+kbVyLGx0ZI3vU3hFb15pYXnMLT+13dAIzrrmhKEHbkayd5qsZAjcfWTRz+5+3QF4NOkt64hP\nTJP9ksIhstTHbljzPNPDoPXnA5D3jizNy5RTeP+CtwEYEb2REb03ApD2vKQUaHdP/bS1GqGiKIqi\nKIqiKIqiKIpSQ3zC9FbcTZIK9Q+u7nPZVBQHZP/tNZueEivjhu5vVR1bU1ECQKeAkKr7PX283DtQ\nI1QaL4ZB5kPiIZ9WIhES7Z+QBETqhasnTHHfJL4kVv/rNt4LgCt4n0122BOy/aA36d6A0HUAJFtR\nLD92nsT8sfLdbUuvAiDlGev2S3WPz8aIPUK22X2w1w9Vx9aVJFjvKhugRIriPzjatAIgv5v0azlD\nZIzSfHwwwTurb9NpLl8r18RbaaAdDqJPqz5uCbVZkWT2g2/JqzQs1780GYCnBl0EQNrwhYc6XalL\nTKMq55s3snbbOaI77X846FUH4Lo254DIXN02ue6ptOr47bw2fJMhUeyRWw+dV/OfKI2TDROOCag+\nPb5n8eW05sCIGKXuiPxWVqC4XxLdKvCU4XpcojDD50rUSbikRuW516Q9XXdHAsuukNxVyy+T10s/\nHgaAuaRu5xmq8YqiKIqiKIqiKIqiKDXEJyJUDsaa79NI/kuEypb/SXbfuVe9aB0J5Y08iUL54EVZ\ne7XoGYlacZseDI9uidXQ2KNkzWpB/44A7DlB7HxnninbDC7IaMkfPT4BoNP4OwBovUGz4TckwZMP\n9Kp98XlCtc/jrrsAgLRbZXvy91v+womBEtXg3UqZ7+TlgqSedVNQ5Yiwhcq2gltGHgvAsMgZVd8l\nBEh02A+f9T7o9d4t7dqP0DwBjYm8QcdY72Y2ZDGaNEbPLuQ9JTmonkyTfcoHBJdWP+nkA6/rt1J2\nr7goaTkAcY4CrgrfU+2cH/bKFpLudRtrs8hKDTHyiwB4Ied4Ho2rnufv+giR2VWDrOjpQdDlvTsB\naD0pHwDPstX1VNKmidFD2sHy+AOjGT6/4HUAPjz5VADmj+1e9V34Djk/8HvZHqbDYolmuDr207or\nrHJQMk+Ssch3RONgO0DV6xHfc/Df5/l75YQveI2OR3VvpWZ4iqtHab6UcyLG3L/Pm+navBWAtvdt\npUuktKfrz5Gcm6azfiI2NUJFURRFURRFURRFURSlhvhEhEpgrlgMM1xi9U90yC494afu8844WrcE\nYM7VEpnS3C4e1sf3dGHpeRKhkvtsWbX7XrO1PwHTF9dhyZUDMGTf+OKLewGQMbiCl078EoBBITP+\n/poW+2XrbiMWS29uB3dBQR0VVDlaosdLFFHWePnc/6LbsN+RCUg+lf1psSCczEFBALizsuqvkE0Y\n23Hibdn8WAAAybF5TOjwGQBxVvvpNmdbZxtV110dtQSAL0O6AXBG8noAvl3fpeqcmJX7zlfqF3tc\nbNUud66duwDYe51Ebr7yxBtV531SmAhAx7cKAQ6yN5dSWxgOGW4VPl3M3C4Ta3z9jGO/AsBuiB/M\nbR4osdbBErG77eTeGPOWH2lRlaPElb4TgN+v7EK38/sC8MiNEwC4PEzGrd7cHQB/3DgKgDGXpAHw\n4+Wye5O5Nf0AL61y9HhzKaQtgW5b7wJgyV0ig67SHfJSizkA2J6YV5UfJdMtc5Fl5ZLH4ZyQg7ed\nL285C4BAttZ6+ZW6wXNaN8b1fv9vv7tr3pW05496LlHTxtOnm/VO6n16eidiWP+P19kKG8a04RMG\nFRZIIqDXcyQO9r/x8vnX4z7lzCF3AxAzfBuwz5Di5dNlvXC+LMsMNp1WXVHybmkO7K2zYisHknGv\nDOyX3jv6gO9G7W1X7fOIaAlbduFmRqkY0dac9gEAp3x+OQCR56pBxVcI/mYh9l/FEJb2/G0A/K+f\nGNPGpMxk0ARZIuS4LhkA1470Bihl08G2VwzUlbulvrfsDmHghPsBiNgmbeauUyWkecWwffp6xoQH\nAGj7gBjMVlrHW6MTuIbE0Uq2EYz9LI8+UTJheP6rwQAsvP5lAEKMgKrz33tUvgtZrttf1wc7HhQn\nwp9d9unSJpcs9Rm5+bIDzh/T9gsAEu0hh/0bD8XK8sr8N4JZco8sVbDP1ElAQ+FetY4kKw/ih8+J\nY++Jl0TWT50vfd/g0Iyq82+JlMnCHT9uAqD/7bf97fJapfZIelYcdqfvlLlE9tnieF3eV5ZkBRt2\nvMH8iXZZyppoGVJsVY6GA4P9dy8Qg3VLNaj4DFuGw0mB1be6WFguMm73tm6BUd8UJwYe0XVmdMNs\nmKBLfhRFURRFURRFURRFUWqIb0SoWCy6y0oO9YVEqITYApj38tuHvGbzWe8dcKzzvKsBaLlJE7fV\nF94kYC/eNrba8VF72zF1RD8AAlfuACDsK7EEeyNUOn53Ox1HSGK3u5+QEDBXCwm9jKzjcivVscdL\nqCvh1pKQjVtqdL13iZZ3q8j3e0mi6Eu+/oDJHWTb5X6nSAhu+OcaoVKXeCOA2o84eD2nlFohl8P2\nHUv6TT01jZHy1nEAvJf6ddWxa2/wRkMEHHD+9f+R7Vv/00e2G2z/Wcnh/9gC3T6ypqROtaJh79h3\nbFl5CwAyP295wPlDBl8LwKxDLA96OLMHAFuKYwGY0OZHAP7b/A/WfiARZBd8cw8A7UZqgujGQNv7\nRA6fvn4KAI8/Fs+qc97423NPfGIRf06ut6I1aaI+mm+9yuczrx4JQElzG6cO/fsoLxuyqcW1cXOq\nlgpVYeqyV1/j/m4/7hd1JIx8WhrsmHm6EUZ9UxpXPebDMP55ExnPqcezYYDMM4enny4H62m8ohEq\niqIoiqIoiqIoiqIoNeQfI1QMw0gBPgTiARMYY5rmKMMwYoAJQCtgK3C5aZp1mpDEsVzWlbb7RPIv\nrLlyNE7jn7dDSreS2Z415kEAUp6RNZNNJQlfY5Bh0huS46Z/sESWDN0yAICicysJdEqUw/rXxEu3\noKV4VTt8dh8AHR9fhqdM1rW2fqTpWokbUo45N0vum843yoLwzBLLHdP/KG+8cMU/n+NHNAZdrAmZ\nvYIbugiNksYox8CtOQB0/vhOfhjyAgCpjoPL79oISZx57aWWd/zSw/+t85N6HFEZGxP1LUPPCsmP\n0fX3a1h+orjBLwmV217yr7+PUDgUQ7cMoEiCizBLJLqo70AZGzUbuZkv204H4MfBkqj/bJfkR2p7\nv39FqjRGXTwcNgxPAuDaXr8d9JzjQ7fzJyn1VaQGozHKMPJj0ZNIYNPLhz73zqlXMvv46lsnR278\nZ2+6v9EY5Xg4GD0loX7noI/wWFFHj++RPi72Y0nC31Sk2ZhkmPijbGLBI/LitB88Otqb9H3r7VTJ\ncO63XQFIYd5Br6tNDidCxQXcZ5pmZ6A3cIdhGJ2Bh4FfTNNsD/xifVYaJypD/0Dl6PuoDP0DlaPv\nozL0D1SOvo/K0D9QOfo+KsMj5B8jVEzTzAAyrPeFhmGsAZKAC4G+1mnjgZnAQ3VSSgtPoWTW9u4u\ncdL6Oxl05ywAbomWnAzeLZX3p89vkpOh3TP1Y6VqbDQGGcYEyNZ/yypcABSMlAzoNvtOHJMk2mFd\nO8l3c9aaocA+T1pTiST6JxpSjjknVwDwXqpsbf363vbyYyMGApAwqma6tfd6iXgpSpH1qk5jGV8U\nxQAQvtl/t4lsDLpYE0oSDvTLFCVKVGBQfRemEdEY5ejaIlGAbR7axl3jrpeDDpFV5imiWyfctOyg\n1ycESn6jx+P+rPLO5VVWj3BZ+IHk1GleTx6fuqTeZegR71rATxFwYs0vT5shiYyiZ4jmNf9yVVVO\nKi8hk2THptIfQujyoeRgWXHShwAM7LMUgE1Bcr036tPXaYy6aI+OBmDn9Z2qjvUaIrugjU2ZC0Cl\nuWT/K6pd7428fuehSwjG/3f5aYwyPFq8OVmaEr4qx3XDZTeZ/Xf4mbhK+rp2lUsbpEwNRWOSoXu9\nrEpZWykrG6Yf+ym9Hr8XgNRnpV20xcrYJuu9KABWd3uP/isvrXZOfUUX1SgprWEYrYBuwO9AvFXx\nALuR8KC/u+YW4BaAIA5/+7/DIXbsfOaNlcn491dLOOs1j04F4LqIDQAc99VI0h4UhWgqIVuHor5l\naLce9i4hkhTo5hXXABDRTAbqca/YmdBmGgBnrpZtPINvk8ApTX15cOpdF60Eax7LvHVH9DoArr1f\nluxMHt6W5/88C4CYyXLvnAskDD12iny+9V+TcJsi2zNCZElCC4d0ZJWmjfd2nCa/1USWATW29vRw\nCctQzdyfxihH97rqCdfjrK1bt445+DXpibK04Pgb+tDynbVyn5zcauf4gyHl76hPGSZ+s4W0U8Q4\nsr5f9aT5XX+X/rFk1z7HUKcXdgHQbrtlDDNlJHMoLfSUlJA6VJYYdX5Ckip+c/VLANwXdI6c5CcG\nlf2p9/FNWlsA1t4pCaFP7SnbVkc4pO+bmPjqAddUmnbrdZ8Esz3isJhdKsue33noEgDC5m9tcuOg\nxtie/hNF85thO756wP+ml3oD+5IQNzV8QY62rmLw/PoM75JLByN2SbLoDnduBpr2PKSxyHD4vZIg\n+vvXRrH8ttcB2HKz9F9BVqLaJLv81oO7exJ+q0jN5XLVyu8fLoedlNYwjDDgK2CkaZrV3CKmaZoc\nxF5hmuYY0zRPME3zBCdHtqe0UjuoDP0DlaPvozL0D1SOvo/K0D9QOfo+KkP/QOXo+6gMa85hRagY\nhuFEKvYT0zQnWYczDcNINE0zwzCMRGBPXRXycPAmkJrysWwfOAV5bc8CjUyh4WTo9XCuKBHv56Ie\nn8kX78pLqVlB76USmtzsAcvztnFDbRfDb2goOaa9LSF3nUruBOD1geMBCLeJlfjqiB1ce+r7AHhO\n/csiLSvwxIatKsIFq6H9vdwJwE2/X0fKWGmOHPj3dsm+0J4ejGx3KUGZpQ1djEaBL8vx73Bl7AYg\n+b+7m4xXriFk6MrYTburpa7PpXu175JYdeD5R/g7ZqVEPbR6XJYejHz8ZOub/CO8Y+OloXTx2m9/\nBeCCUEme6F2qs3/0ycFYUymvV0y6m9Dt4tv0Lp31LvNpKnoIvt2eNl9Sud/YRriwr8hwZUMUqAHx\nJTluGyRL844J2DcV3lMm0YHuvOwGKVNjoLHJ0LuU9RxGEHn3dgCiAmQcGmyXhnTBV5KANmnUEszy\n7fVVtGr8Y4SKYRgG8B6wxjTN/XNdTwGus95fB0yu/eIptYHK0D9QOfo+KkP/QOXo+6gM/QOVo++j\nMvQPVI6+j8rwyDmcCJVTgGuAFYZheDPaPQo8C3xhGMYwYBtwed0UUakFGlyGX82WLHznnS+J2cZn\nyTrFrU92JOaHRUDT8sYcIQ0nRyuvSXsrP94bnS8AoDJG1i3u7LtvreTYG2Xb6xOsBF+nLr0KgOIF\ncQfcNmmmrDVvPefgyTL9jAbXxZrQ+juJQOpkSB6GNpNKMRYub8giNRZ8So7K36Iy9A8anRy9OVEe\n3HHBAd8tWpQGQIc3xMHbdkPTzLHxFxqdDGtCyMJNvJrbGYCRMasBmPqd5FBp6ad5pw6CT8mxtMWB\n8X+bP5UNF5rRZCNUGq0MQyb9TqUVL5P1l+9aWHrWkCtSDmeXnzmAcZCv+9ducZS6QGXoH6gcfR+V\noX+gcvR9VIb+gcrR91EZ+gcqR99HZXjk1GiXH0U5UtqPEC/M/0YcZx2RLbADWNRAJVKOBvdq2UHC\nu2YwZc6+7/79TPW8ADGsr/ZAOIGRAAAgAElEQVSq+A62WbJDWttZDVwQRVGURsaLLwwB4H+hMv8o\nOVEiLtkuuxi2efjArXPbIWMhjcj1H9w5uXzy/gAARt63uoFLoxwuP5/nXdESXHUsakNFwxRG8XkO\ne5cfRVEURVEURVEURVEURdAIFUVRFEVRFEWpAbHvHhiBojRNEl+WHA4XvNwTaHK5U3yKvGtPAiDO\nXl1/H9/Tg+C1sgPbke6upjRd1KCiKIqiKIqiKIqi+DXZ3SR1aYgRUO34Dx+eTEK6GsKUI0OX/CiK\noiiKoiiKoiiKotQQjVBRFEVRFEVRFEVR/Jpjum+t9vmJPd0ASP5iqy71UY4YjVBRFEVRFEVRFEVR\nFEWpIYZpmvX3Y4aRBRQD2fX2o0dPHNXL29I0zWYNVZiGxk9kCCpHf5CjytD3ZQgqR3+Qo8rQ92UI\nKkd/kKPK0PdlCCpHf5CjytD3ZQiHKcd6NagAGIax2DTNE+r1R48CXytvfeBrdeJr5a0vfK1efK28\n9YGv1Ymvlbe+8LV68bXy1ge+Vie+Vt76wtfqxdfKWx/4Wp34WnnrC1+rF18rb33ga3VyNOXVJT+K\noiiKoiiKoiiKoig1RA0qiqIoiqIoiqIoiqIoNaQhDCpjGuA3jwZfK2994Gt14mvlrS98rV58rbz1\nga/Via+Vt77wtXrxtfLWB75WJ75W3vrC1+rF18pbH/hanfhaeesLX6sXXytvfeBrdXLE5a33HCqK\noiiKoiiKoiiKoii+ji75URRFURRFURRFURRFqSFqUFEURVEURVEURVEURakhR2VQMQxjoGEY6wzD\n2GgYxsO1dW5DYBhGimEYMwzDWG0YxirDMEZYx580DGOnYRjLrL9zG7qstY3K0fdRGfoHKkffR2Xo\nH6gcfR+VoX+gcvR9VIb+gcrxEJimeUR/gB3YBLQBAoDlQOejPbeh/oBEoLv1PhxYD3QGngTub+jy\n1eH/rXL08T+VoX/8qRx9/09l6B9/Kkff/1MZ+sefytH3/1SG/vGncjz03xEnpTUM4yTgSdM0z7Y+\nPwJgmub/DnauIyDkrMCQGNyxbgA6hmazoTQGAJfbjlwv19iLbbgDrOu9x0rl1RVi3dhh4syTt7bm\nlfJdnnWRx7qm0iQwsQyAokr5zpErv1UZYWIrN+R35YVWcXsA2JLfHICAfBNXnNzMLLFTmZeLu6TY\nONx6auzUVI624JB5zqgYwiJFGCZQ6ZH6jHBIPWeVhMv5FQbtYjMB2FQSB4BjjwRFVcTKPZ35RpXQ\nXTHyGhUo9y7eKYJ2NfMQ6HDJdd7npMgh10dUEGiX7wpKggEID5ZylOTKZ0+YB1uhFYxlPUul2enZ\npmk2q0ldNVaORBedkUFnBSdE4NntlC/iXQTbRYdC7eUA7M6Olms8ENcsH4CcilAAPAVS/6aIA3sZ\nmN4qlq9o1czSpWzRJWexSUWMVQ6bCMIolosCCtx4kkXP3HlSJneInOMoFHVzR3kIsJ6D8nInrpy9\nuAubsC6GhMxzRsdUtXVmgEnzsAIA9lg6iFuqx1Zh4AmU+rSXyjF7uXyuDJXPgXvdVESIQD3BlqJU\nyneWamOrNKmMs74rF9l55e4MqiTI0sUSl7S17jK5n7cND9xdhqu1PCBuj1xYvnlXk9ZFe1DIWc7I\nmCpd+juCw0UnK7ID8URJ/+kpt+pWPmKpLfYKDxXhlmyCPNZvyXe2IlvVNd4+Ly5edHtPbqR856Gq\nnYyMLQag2JJnRanoprMYKsP2lc+Vm4u7qOnqoj04ZJ4zMobASEtOuYEExlj9UGkgAGEh8rk4P7hK\nH2zhoi/e9tRRYulkuCFyABLi9gL72uOAPLkGlxtnOzmpqCC4WplMG7SMygZg+17pe4PCpGyl5fvG\nSIZDfs+02omKbTubtC46I4POCkkIx201agE2F0VF1es2oNCSUYhR1dcFFMgxd4DVV3nbT7tJ4FaR\ne0WiNXC1dDIgS86tDDWwSddLyxYyXtqcGw+ALdhdNSY2S0XfY6IKAcjLDq8qk8fqxm3Bbioy83AV\nlDRZXXREBM0LSoikolLqKy1sD+sLpD7xjjusvis4vJyyfNFPb/trWjph36858+qrK/wvYxK5FGex\nidlc9DIxUNrT7fnWmDfIhdsar3rl7L1fSPMSAIJslRS6ggCoyJGbNvUxqi0s5CxHbDTNwuR5z94b\nUfXs5+bJsx8XJeOdrNIw8IhMAqw5oVcXY5rJObl7IrBHiwCSg6RN3VpiTUSKRPg2F7gsdXcEizzt\nO+VZcQfZcEeJ7nrnE169C9xjtftxgfvGwTarXyxuuv2iIyJ4XkB8ZFUbZgBRATK/q7AUrjAvZN81\nXnOE9eoJsvStSKrQ8OwbdxhWN+gMt+b+BSKM6JhCSi3jQUKA6OI2y85g5DpITsoCYNd20c/4lFwA\n0nPlWXAWm3gc8nveZ6Ei/fB00fFPJxyCJGDHfp/TgRP/epJhGLcADwERNnsAXfuPIO8aUYpfe77H\noFVXArB7ryiI22oEwxcGU9RSHl67ZfSIXiOVm9Vd7m3GVpA8Sf6FwLsz5LvJKfKPlcq5obvdtHls\nDQC/p7cEIOpLkUjGmW5CNosQvI3puBtfB+DKabcDkPq9h6wbpdEzl0ay9d2XD7uCfIR/lKMlw1uA\naFtAIK1uupcTz18BgNs02FMqsuvXbD0Aby/qA0BgupOJ170IwMVLbgGg+ZvSaWy9RuSTMC0AR7nI\nefcQGXhcmCb3XvTYCQDk3lpEuxgZHG7Ll0Fl+VxRhuSzttEyTBTixz+PAaDvsesAWPppFzn3tEKC\nfxWZ2ywlXDrmvvQa1FFjp8a6aA9ycsqYKyh80dKXkbs5Jkp06MTwzQD87/0r5LsSuHn4twB8vK0X\nAEW/ygClIlLkGL3GrJqYl0fL6/jhrwJwzXsjAYhfXMm2IdZgMqRCXn+XZyfplzxKnxc9K/wqEYDc\nXtJQJvwsOp53cTEtY0XW6za2YPczr9WginyCGuti8t33VA3uSltWMuKUnwAYvbwvAO4C6VhCtzgo\nbiv1GfWntHmRW+Tz7hOlfltPyid9QBQAxceKLtozZHAXvVZ+P3S3i53XWzP3rWJcc4WITJM77KFD\nlBjRFu+W56pwveirXcRN2xfXkjVK+qX8QumtNg35vyatizZnIG2vuZeKiP2+9w4o7PKmS98NAGwb\n156KC2XEWLxRDCABeTKCi9wkcgjfXsaOMy1jdAfRKbtDrC5Bs0XfAgpMPFbvf8N93wHwxmeDAHAU\nUzWZP+va+QAsypa+M3256GbzxSaZveUc026S8fyof64Z36JGumg4A2k17F46DLTk9Ek72l4j/eGi\nP9sCcFo3UaLF3x1b1Q+F95UJdNEv0p42/0N0a2ffgCrD56PXTwD2tcepX4uOkVdA4niR75yfu1Qr\nvDsQXhv8LgB3fnETAJ1PkXZ9+SbRTSpsOKNEzyuLpJ3YPuzhJq2LjiAHp4+9nMJKafeSQvOYN7ez\nnGdN2JJmivD2dHdSFi96lfKj6GlRCxlI5h4vxx2RFbS9Qcaf227tAYCnU5Fc85YoYGavIEJ2y/Xv\nPCV95uWfSZ8ZesxeXJbh2bVc2ubLB88CYMrYPlX/Q3GSNd7tvJcN9753mNXjM9RIF+1BTrq/eTU7\nsqTvmXTKaM6cfjcANmuS7Ngm49AufTawZloaIA5WgErLORvzu7Pq/l6HQtYZ0pHFzRR9KWgjx+MX\nunHfKWPUx9tNBeCOb28AoHnHLPLniH4HZ8lveA0r3W9bBkCn0AxmZneQf+4juenSsU17jGoEBJDw\n+N3cfvKvAIybeDZDBs8E4NMp8uzfeNHPALy74hRcpaJPqV+LvhSmiC5ecbuc88XrZxJ+qYxxX2r/\nBQA3LL8OAHOuPCshmSY5XUVGzTrLxDvyX9KX5rcPJecCaW9DZ8t8oiRRzm07aiMA24a1x20ZAVwh\nJjtfefXwa8g3qNkYNcjJsa9fR3ml5TCwe7ggReZ3u8qlPZvxrUzoDfe+cYdXP4rS5E3z2ZYDvdRD\nxqmWAzBb5BzfZycA2T8lAXDpVTNZUdACgEeSpwFw84prAAiYEM3zT78NwL/uvBmAe0d9AsCDn8mz\nkLDARXG8/N7eY0WWW+69/7B08WgMKoeFaZpjDMPIBQY6g8KGuZ0GV7dbBMDxU0ZUnZf8k1RSyY1i\nOQw4t4AKqxF6+QbpILwNlMfy6gw9bjHH9RbZlpnS+L3QR0alpavltTjJweURWwFYZEsFIPsisZDZ\n00MoaWmNbKyB6yObLgbAuVeElXWcncoKqabK1ErMgKa3zbRpmmOAMYZhXBpSFvBlqwm7yThD6ndL\nViyBv0vj8lGADLqvHzoTgC/X9+XcOXcC4Lash3tOsAxgm+Tel/9rGmPWnApAxA8y2J/kOh4Axyly\nbti0KJYdJ7/Xq5sMWHPmy2TOHGBgt2YfhuXB2VIglkbvhMG1JYx2V8ngdvEmKSNjSDm6WvE99tdF\njzNy2LqFrXCdLwO/kF+S2dxG9G35lG4A3PacGFHe/mAQo7+QyVbXATIp+DNQzj174GIAki7MI84h\nhtLxj1wAwHM7z5EftuzznZ5eQWChyGbLXNHFoGyrQxq7iZ+3yMBm4C3SPqSXSIO7PLs9AK78QNLt\ncix1ikFuXi1Uio+xvy7aIkO+rIyrxKj0uidN3v5K6jxG5k5knW4ZUTa5cVjRBfEXbwMg+1ORQctT\npA094+I1jP/4bACCNsiA09uxXXCvDGo+WHES4cEyqDxjwEoAvpktRrYdO2I5sdlWAPIyRZcdSVYk\n23Yxnuwe0pG8TZYRrk2u999q0rpoRIYOK403aX6cTK5dn8Szp49UfOendgOweafoQEClSftYGehl\nBMnk+5JkGZSPnjEAgJwLPbj2iF4HrpfB4KnnLAdg5ubjAIi9cCfxwaKvY96xDCneaLM+udimygDz\nz70ySMmfIgMUd2drQnJjFm2flDY5v30IWSW1USu+xf66aA8N/dI0YNlWeZTNrm7KpsoEqdVSkeXO\niSLDBFs5g0fJID/SLhFAT3c/H4A9NpHX/10xgeffFgPK6M19AQg7XQwpa1qLbCLinXRyyiRu7JVv\nAfDQY8MBKI80mF8sv+d1SLUPl+vL3hBnxH8mvl810CzYWeUlbNK66LFHDVvzW5uq6Mjs9GSirIiU\noDwZ7e8Y4I0M82AGi54FjBTdDRktehLUWuSSEp7H4v9Kfxq+RX7vymPmAvDlA9ZE4pcgPJfnAPDy\nbtHh5JnyzGyNjCQqRTq6gF1Sjs+/PV1+40wZKxemRxCUIM9RQUEwbrffOMQPm/110QyK+HLrpnie\nO0MMkReNfhDaSrvV7pqlAKx/W/qsjV+kUd5b+qjQhdJHVcZI/Z12q4xDkgL38tYfMoEPXiP9YlYv\nuV/4Rmvi1cFBoEve371oCAApP8uzYfwQjduaeGf1lmO2UJHvttvF2PrT3Z0IsSLIIguq5hhNWhdT\nW1QMe6bPRF58UeozKAA+mC9zhU6jZQLxaRdxuhrbg7ElidUr5VExbizNkL4rzC7H955cQdEicQhc\nuVeMzMFzZe5Sfpr0hSkJGVwcJeOjsX/Kb0XnS4RL5kAnL3T/BoBHN10FQIvZ8hxsu0na2pi1bjJP\nkLnjCSetJ29sWa3Uiy+xvy4azogvc5Y3JzBHdGpvWxfffN0PgNh3xVnjvE+uM1wQuVXqM+tq0Umn\nS+pyTz/RiW7ttpGfIw654PaiQ9elyH2e6ylj1wkT++I+VozWN+VeC4DnV4lQibt5C9d/fysAQd2l\nHR/x09Xy2TJ0Z9jCuPFC6Z8/Hj/A+28dli4eTVLanX/5kWTr2OGcqzQeaipHfyG0oQtQi6gu+geq\ni76P6qJ/oLro+6gu+geqi76P6qJ/oLp4CI4mQmUR0N4wjNZIxQ0BrjzUuZWhkNXDYMIWscy3bLeH\nzLli0U8/34oJz5Aw5vO6/clvPSWs7vnhYkGKbm2tOQ2R4zOS2jN1umWxvEw85wG/yvVmP7Eq2hdG\n8M574oH7160S2jMzvxMArbtkMea7swCItCImdu2R8jx9xacAPDzvUuzWYvOwDU5sZX5n/a+RHMsS\nHax9LBpzvVj34pLzyDlOPN9Grrz+ltVOzu6VzyOdfgHgpU8l8if+TImc2rZUrMcl7kAqtoqVOPXK\n7QCU/yBRJG7L8V7SAkyneIc250mEw7vjxSP3R1kqT0+Te4dvE/vgtkCxYN5+43QA3px/Busmircw\nsl+VV7z0sGrHN6ixLmI3ccVWEj9DmoCss0uJmi8emu2DRRdj7WKxrYg0qYyU+l8yW+ox/lTxnM97\nWzwE190zjVffEzm0vkfCI7a+KxEngdaax1+/70bqD+LKPv/13wF4pPlsACYVtWdqoYStL/mPhEYH\n3bFLXtNkHWRRdij2WaLfvf49nw1X+p1bvGZytJnYAt2EbxdFSTtjE45jxAO26xHRweselCVA7/4x\niKCB4qHe+b3oV9BF4kX1/Fv0Zew5KbhSrIilRJF9/GjxyL3XWryi0cvthO+Q3/s1VdZ8XDRsAQCT\n/ujBNz/KsQ4TpP3d8rC0Ez1Ok/Y5qn8p0+dKBFrz/6vqfpq0LjpKIWYlFLSXujYHFxAzXaI/Njwv\n7V2beGkbu0bvZOJMqeOw7dLevdNb+kDvuuPLOy7l6+WnAVB+rFTtsnckMiXGCsjMTU+m5Dzxqnvz\nP5ScLzK7M+03vokQGZU+K/1h8fXiAU+IkuciuzAUx8nWkq9gqvKd+RE1kqNph4poDzcePw+Aj9b0\n4vvz3wRgwIcPAGDvKF5Qj8fg42dl04DAQtG3ROtGe7qLLKbnHkPcedJXZsxMBiAs3YrsstZ1l8VG\ns/DDngDkPiiy6DxCosZeTv6JU14X159hqdm8Pa0BCHhWZLiqvAUlZSK4dp9K2bY0cV20h7iI6JZD\n7l6pz8KAAAwrj1RQtrRl7Y4TXdwxI5XE46VNLaqQeswdItXX7mbpQxePSKvKXVXeV/Tru4f7A9Dt\nXxL6Pt9ohnuq6PmCVHld/4GMb1pPvoXyBXKs5WTxvBe9lgBAdIj8Vll+FK4CiQhM/MNDVt5Rbd7Z\nGKmRHG3lBmGbHLycKt7li6+exReTJMLEFi71FJ8q48Cu3XcyY7NEF0SfJ+MNz4/S5s1ZKLoVdFkm\noSulbe59iUT6edtTm0tkW9DGIPgLkZMjVZ6XS56XpT9Ow8Xra/oCkPaCPENZj0o0ypbB4jlvMdlD\nWbQsMytOqJpjNGld3JUTw78/Hkra1TI5WzW/DVErrcbsc3kt/V0ilsO2Qb8BIpvf3pAVKK2v3QrA\n1D4dAbCNcnPrhTIn+GiTRCiFpVtRRAES7bBkY0uWlFrruKxcOm//PB6A2zdfxspSaYu9KSJOf07a\n+0mbuwJQWBRFoASOUTgkBPfupq2LhgsCcwxKWsj84cKef7ColURGb25/EgBBaRKB5/w+ksyeUrFR\nU2TSsOc0GbC0/0Da06AXXJQUi544fhbZv1FyCQA9b1kFwLbxHSnOkPY7JEv0Nv8GEUqIo4KwJGmH\nY6fIORnDRBfDrRxnua0dVVFNjn0aeFi6eMQGFdM0XYZh3AlMR7L5jjNNc9WhznWUMjV6FRRZYf+x\nM0qoHGzlVMiQQfqoIeMAuO23a6oSV5rdpALLuskE6ox2kiNj1tZ2/Of2jwF4+BsJwQqxJuGlVkLS\nlP4ZlFoh8Q/PuxSAW3v8BsCnm3qS1F3W1O1MFeFE/yjXPbxAhGTLdmIi10+963ku+D67plXVqKmp\nHMOjkkn5ys52axOpgmWxxB4v4arNH5OBWv/pMnl664+zGf2zTLJLTxCFKH1XOquJ/5O1hVtdsbTu\nJobMdZvku5YrpXHLbyP1nte9grOOlSI9liATxL7fymAxYp0dK08YV90qjeXqIrnP2z9Lh2qLK8de\nJo96Oytc/s/q6wB9miPRxYA8pqZ+Y2PHmdbxYifFqVKRMQuk3p/aPhSAytZlpCbKACT7F6lb5ysy\nEHDeL4OQ9oG7qxK0rVhn5WWRfgxPoOi4J8JF0stibJnyk3R6rQbJszNqeT+cofKMpJ8vg9O4UkuH\no6TBXZMZRsWpMvDfVBRHmafOVyzWKzWVY0izFKJnBlE5SOpn0zsdKRksxqegNiKMNz8SY3KACTkr\nxHDSYo10UqV9Rd577pHOI8xWQn6BhP6XbpeBZ1GSDO6apUi7V7ytGa6zRQbNwmSS/fUqmXzbgl24\nrKTDO/8l9w75RTrGLTliiCtMtWFF07P9XGlzWdq0ddGMdk2tvDSXPolbAQh1lDP/Ypn8HhMh+jFn\ngxjIts1siaVOOC1DSNkGy3mSLPqzLC+5ai1y52Tp33YEyyDRq6OxqyrYtcRKFt3cWm5bIIOP57+7\nkLjlcu/My+VZSZkgF+a1kd+qTDCpjJdzmh+XSfrnlTWppkZPTeUY2DoZT7MKJnx0BgDJS8oZuPJB\nADydZNxSsVfaMyPAzYkjZKnkj5ulkYyYalmdO0ofGuaoqMppVWL1i7vPkjq2Wc4FZ4CL9GS5Z8Ym\neT5sVm6IC+d0Ivl+CV3fvFAGsJEjrESKYXLNh+EX4D5X5LrpCuufWdzEdbHAMdX1QxzxWVLHhS1t\nJP8gfV9uV1lqdVozMWzMHw9rEsQx5F3W2vZz0cH234jDYf1PKZjR1nKTZtKGbuoiuvjTApmUm2mV\n2K2kxAEFcqOuC6XvjVlqJ+IyGR9tP0Ha78R3RWZFLcQMF+qEsIvk97JLEnHNqWFFNXJqKsfghBRs\nLqhwyeRs0sd96H/5EgDmZouzpihPdHLurG4EWynBdkeLDjqsYcXV934PwOs/DaRZXzGc/bJW+rEg\nq1+ssBxNps0kMF/e5zvlBqO/2W+n1XbSV5Y+JX1neZH0syeeIf/G7IQ0opZYSTp7Vc3dmrQuBhR6\npib9Vsr2njJOaPNNMRuutKzJE1sBEHimjH0S3y7i57NFNrYQkc2mLCsZ9xUylhnccT7vThgIwPLh\nkivz2DJJI9E3UdrKGaXtqbSSvUcvFTl+dpIs2dtbFkyR1YFWRogh5sfnxHFR3NvKjdPSg81a5rwm\nNYmy5/zL01DjfjElhbJmJqZT6mfl/cfR8hlZ+5iXV90wXF4ayd0XSz63sW/JuNWbRPr28V8B4DZt\nbCmQQWZ+P+kPC8tlvrLqA8mhGTVyJ2U/ieErVHxG9EuRVBFz3uyJ8yIZU1WGiZxbvizPS+SL0s7b\nP44lv4uVg25fvtzD0sWjmpGYpjkNmHa454bGaRRXY6QmcvQj/GoGUFNdDI9KruMSKUeC6qLvU1Nd\nDGmf+M8nKvWO6qLvU2NdjNcxamNEddH3qakuRoQn1XGJlCNBdfHg1KuL1zTEQ1aaJBb7Tq+uIv1b\nCauL3CjW3du/vx6A8K12hlwny0U+CpXwrDYxErYz4xfxiHoc8NAOiTbyWFvRYYjVKWinWK1u7zOT\nJydIUiOsLa++3SlLC/J2RRAxVUKrQ62tPPPPEQ+RWSyWxcfPn8Rrr0q0ymf53ch1z62FmvBd3AEG\nhcl24lLEymek7kvSm3muLCV4e4q8Nl9mki2RcKROEu/Yl2+9BEDvKfcCcEHvJaTnigU6fI0VXmRa\nlseTxHLZLKqIlkFiPbzyXolMsVnLUqI2OdjRX2T+1q8SbuFoLtfFLBfLoycgiKgNcv6ilW2Ptgr8\ngoikIgb85zcmbJLld08fO4VX7hFdMtxWZJhNdCJicyDbeos3m/ZSjz/cIRFGZ78lXtjwm0txWitw\nLuopy3nm/p+1NGGFeFo3X5fC0mWie4aVpG3UNPHiuMPdxLcSD96uUvEEmpOt5MKL5blITYKiJPEi\nLevShlJrO9KmimmH8hiD2HekTlwjMnHPFqt/Tndra/pjxLC+9edWuBOtHUSGWksHrKWMBbkSdRCf\nkEfHRySCa8u14tWuvEz0vGCNyMKIMnF7t2j9TeQU2Ud0s6QsgOAV8l1pc1maVXSstPVt2okHNfSZ\nOJx7RT8z+kTXSj34Ou5KO3szIsiLk7qbtvoYUiaK7s25RLwxjkCpR8MNFZ2tUP/m0kdtvvQdADrO\nkQSjq7cnEmE5OVdsFMNpp19FrjsGiZc7+7gAIjfKcxC7VPrV4PMsD/qf7SloJc9G68/kHO9OUK2+\nEDnm9I7HZrUTxekJePL37YjRFAnIhVaf2Miwdj7aO7KIcmsHwcA14uYyrV1E3OEGP2yQnWPYJt+V\nXChhyBVWlNCesjCKRkri7/yB0nc2/7W6tzP3WOhxsnjesv8rEU3p/a0lCKkOEgKt5PvWMuVWH8sS\nohX/lU55Z1+jKnHmpc9KdOfIB46mFvwAEwyXSUErqfMbr/mBN4+X5Y6BK6UeJ74rUUgMBnuR1HfY\ndvku8z6J9tvxnYxZHZ0LcS4XD/mmKKs/s1TFuztXeaKH0PbiafduJV+yXsZESekuij+RCKXS46zk\nuJkS5bD7emv3nx2h5K+SZ8VsX4YZ2PQ2TtgfTwAUtnZjpksfdPM1vzDuR0mEGW8l63YtFb2LWeti\n+4UyLwjaLvqVdq7o1PoS6UuDMm1kBko7HNVC9DRslQgxp7OMPcuaeagIs3YdWWT1eY/L7k7rXj2G\n6Pfkug03ifE8sJNEkqY/Icuio9oFELlVxr1drpElLluPvip8mvIYg41DnfRtLlHQpS85cf4qUShR\nG2UcWlwkMl79SChRhhX9YwW+xkfK523WCoeJC3vSYq3IP+1H2XW04yhpI1emSZtYOdBFQI7I1LuD\nU6W1vidzZXOWfic375Qu/ema+6U/jZ/t3WmvhC0jpC1o2XE3OcF+ZROrOU4PnsQyYmZJv3bGqLl8\n+6zoYvMckWHRKZYuDSjnxd9kUwXjBCuZb5nU/aPLLwLA7bYROkPGqxcPl1C8n16SJc/edjV3UjLF\nPWWsW9xSrs/4SewMLa7IYPcC0UG3FWV2+uOS1H/aOLlPiOlh/LfSxgfUMMOH3y3wUhRFURRFURRF\nURRFqWvqNULFEVNBs1HTCsUAACAASURBVCHb6RspHutJS7sTaeUIzT7PskhVio0nep3Ju7P7AlRt\nTZc/Wbxtg+4RD/hvo0+koJ2YkNp1E09sYZpYIyu+EIv9Y9OuIKhUzjHjrbXHdrlffMtcXCGyzm74\nbZMBeGvMhQC4Wsm5b/33Ery+t6GRS/nM7neJMGuELaqSkAszybU84c6ee6m01qred+/XALz6vuRN\nMdwebJY8s2+QNaSnfHq/3ChU6nfW+72wW8vHT7xCkkr9OVrWFndvKfk2Hk2axuAfZftlo6+c67DJ\n9af+ZwG/PH8KAD3v/QOA38aLNTK7t7VlbEIhWzeLZblDmjwn24+yHnydnPxwPvyuH84O4jm5f8rV\ntM4THdx2vnjKHUWiN2UGjBoguYrunyh7tZ82TSKMOk4VBb4x5A5iM0Qmf+ZLqObem8RD4Aiz7rO9\nAvNYsRwf30K8Dqv3yHNkLoqkLMlqjqxtH5vPEG94v2/+BOCr7d0oXiSRMqf2WMP3If6Us63meAKh\nuKUbW4W0UOUL44k+VRaNVuRa25rPaAVAh7M2kVUqlv3MlVKHzVrIuaFfipfnkhcX8dEZEjH02LWy\n5eRHV8tn12mW5zTR5LIuome/TxA9y6sU713CFhd7xeFGZUuRc+xsaY/Ts+WZSH58J3u+lPedrxAP\n3opRR10Vvo3HwCi3se5dSZbubAO5nUQHQiMlYrLF4NUAhM+OY/lcSaJosxLjtf7uZgBiF4n+OItM\nQjNEl0sSpf7X3CvtX1yiRKHsXRtDRYSVbHOonLtpp3jCB161iG+Xi8duS6rIvf0H0n5vv0T0tey4\nElLHSQHy2gfisddKTfgu8S4892dzcwtpqz4YN5DofCvZr1QryTPFc93vv3MZ/5usvQ/PEBmc0FfW\n8G+ynoEsWxuMBCtHTprIJ2mOfM5/UJ6JwNnN2JYvume/S6KMjg+XSIfNabF0CBP9tg2Q65a+IpG9\neceJTANbFBK8S9rsV5b0t/6RX462JnyapPhsnr7/fTaXy/jxj8JUAlZLNENZnNRVpaU3houqqL+y\nUmubebflqbaiv7JaO4nZLe+9vZU7SD4nzbK2Rh5so3S16Kc3muiBobI966ezziPrJHluOrSXPrNg\noSxLOq2lJLWdvaYrbV6TvHXbbu3ojxsn1BzTICpJokAmjOtP/E4Z85fFiHy8SbR397YTsFsar/Jm\nck50gEhq0x3Szg5/fyrf3CXRz7H/lsiEjfESXRmSKbI03DaKLpHfy94sEUkdbZbcRq5i5lLR64AY\naUfdi0XeW8+X3+zdYw35V8hztudOXXYGSF4iu8lsKz9U24QsotbLV7ZKS79Okvpr1243Tpu839RF\nhLtth8ztAnbJ+Kgy0lOVW8wZZEV8Zkh/WNlVxkBxc5w4LpN8Obt3SNs6brYkNDYj3Gw9z0qI2lIm\nLYGWVmeeKmPmkssqCZwjz8aOxGAqSpp25CYeA0+Jgy43SbL0Tz7vT0yZtXHCqVI3YVMkcs+WZlYl\n8G79pbRhO/pbY5rV1kqSApPwndJuTv5Y+lBvRvfiY61cgBGlONeKfnlz3YR1lnnKrj8SGXSe2A/+\nzJNx6NIbjpV7PyvzjZ7NtrP2JomE2nlmZI3+XY1QURRFURRFURRFURRFqSH1GqFSXu5k3eZEMlZJ\njg1bqgdnsVikwueIddZxrlgM93SPI66VWIPtVjTC1Y8uBODPIolUuf6B73jt6/MB2GlttVvUQdZl\nBVipMoJbFnJRP/EaTVwvHpqsQrEutorJZXuyeGs/2i67jvS7Rn6j1Nqzd8X8rnS/fykAZ33wANtz\nXqmVuvBVXG47e/LCaD9ePGqBZ1ewapZYkCc3k/p1FopMK0NtnDxQ6j79blnjXdDWWuPdSrwC4+9/\nmQun3wXAz6tlx4NB1g4Ic96VLXn/faWN0GYSGdSzhcSWLB8vVsX1Kc1xWwvdvpsv+UAizxRrZI8Y\neZbS32nHQ49NkTIO7FE7FeHjGG4IyDcozpSohYDkEtxB8szHWjt85FrL/JNmV/L2xPMAaBMiES1B\nL0vdLr9N9M5RAJ889yIAg1+SvCplJ1le7XWWd3wdDL33VwA+f/FsABK2iodvb5pJ8VKxVAe5RJ7l\nb4t1eexKiUBq+0wFRaLurB53DKXZP9ZGVfgstnII3WanOFXax4Rj9rB3lkQQGF1EX665VDzOX2/v\nSna65QW1HJjtwqV9nXam5Et5fUUfgmLly2ffk20/yu6R+7jKRU4922/li2WiQ1vGjgXgtDtuBSCz\nl73KA3Rn95kAvLtBsuq7Lc/D1hUtMI+Rk5Z/16kWasH3sZdB1GobeeIUIbXHTnYsEu9J5V7pF1ss\nEK/nrHWpRG2V87zryFs+I2672fnSJtrcBq6rJIrBWC4etSBrq+uyDeK1s/cowtwsup81T1w8lUni\n+Zn9fk+MzqJ7gVnSTmc/LHqfFiVe8sL/S2brrXKOx12K6c1h1kQxsxx43mjOW1dIvg1PqofCcPGC\nNksQz/Xwq3+Wz44Cvt7QF4CeV0lU5i8LJLeU7SrRs9PabqR5oET43R4quyL8d7HsZDili+xQ8V7y\niewqE50udImcV02WPrTDoPVM+kg8qxWRonvNrpWIlfL1su7fzA6mxMpNbt/d1EOMhJ2743jiuRvw\nDJIxROHaGALkMaeZbBRD3M1bAVi1OgWb3YpQsLYjL7W29Cw9U+R4Tdff+ThXPKmhdtGRuy6Sscjr\nRRINfVKXNaz7UORW0Efa21EfSs6AitNMIlZLm3zqSZJb48sh4q39eaW0n7YID2v+I2OwxBke7OVH\nXw++TECBSep0N9sukHp65NZvmHjTWQDktZP2tCJGZBHaNp+u8bKL0uw1El6ZVS7zg/W3SaTD9vIY\n9twlkQjb58k5yRfvqvabrYKKWTFLIlqsR4LZ38ruMBFbPDiOFRnad8q9z7pY5hlOQx6uiQt7Yn9O\n2t/kj/1r98IjxVlg0OInO7mXSeRB1pcpxG63oqiHi/xCVoi+xXYpZm+ZyDYgQJQxIl7Otc2ScWXR\nhUWMe3kMAJf8IVGdue/LM+J2S66445plMGu9yDFitfSZFVZOFnegjYA8kWOxtcX1mV0kynbGdmm/\nQ76MxHOljKuajYokq6Bp5zOylRuEbnayuLlEXfUbvITf35J5WlxP6Y/ca2UOfn6/xWwslL6pcLqc\n704RGZ55tvSTs946kfJ7pW02Z8pY19u/xTeTfnbvwng8YdZ4xFrdUrRaoo2C9hgs+T8Zv+65UXQ6\n+GkZR3mmyP1m2hMovUTu6Siu2f9br5obEVLKOV1XMj1IOoKIBcHEXi0T8535ElqTlykPeOt55Wxp\nLe+dmdKwvfmbbKV07mXzAfijoCVmW/mPSyplQGDLkQc9ZJe1zGd3JF+ukWQz0d0sA837omA7r3YT\ncqaEdxWVyXXfzZRJfNRaS3E6GqwfKeUNOQ5sTTzHUKDDRbv4bDbcKRPp1H+VE2o94Oc3F+PJqGjJ\n5hSa6WbWbGlokp+RZV7lk6RDie4j4VVXvn0vAWHy8J4+QK7/dokYZsLOFgVZuro1CS3/n73vDo+q\nbNq/z/bNZrPpIT1ASELvvReRoiCKoCCgYEVERbHra8cGqKiooFJEpEhVQKT33ktCCOm9J5vtu+f3\nx/3A+73X910/9SOXnzFn/tmUPefszjwzzzwz98zQ4OkEjNIRSvkcT2uKng8R7uoopULUWgm/y/+R\nTkZNMxU+2kjnRTOXTiruunleNGSS/D3Q9ykDDvCAFbVChWt3cgO5ZzAbL/+ymI6gPVSDzLu4qxjF\nwUxbSYMVeJ4mxBEOPDp+OgAg0irGj91J+clP8725E5tj0VoesOVmlHlAFj+PPUKCOYt/e+9lNtl8\n+/GpAADPbQTSFfX3h9SdkPa6s4Hw/bMm0v158vdC1bsSkctoJx3JGng7cn2HmumYL9vA5lreRDs0\nVbSRGht1J60LjZnmDfI3JbIY6Xrq5/UGX6bD1+Hu/D1rTxLQl9e1XkB560RJQ+AV3w0o9eLUXgCA\nsDPU1/x7uWlprxqha1ctbqY0pQUAn9kHa38b9GcZ4FC9FYJhcxlUPlKcAAA4v5jBErm3C3Yx5tgZ\nzD0r6xgjn3IAeR0RXw55CZ0UmdWTcDQVpywxErLJBj/URfE+1wPgzqbU06p2bkiiVNPZjA5N5Kdc\nF1dn8O/aFAMi1vH91ig11HWNG+zqMUko7qaGp0YogMmLcZ0ow2Nl3Cu/yhHNTdUerH7mQwDAlBfZ\nZL3/TE6ePPoz98s99mRM7EJo8sevcoTue+98AwAY/QED1jXd7bg1hQ79qb2MxrmTqGfpPyVh2GT6\nSVVu6nDWszwMdpnDsb/PRm/D7C2PAwD8CpUyEQCQNYA9VIL7Ih1wU4GE2u50vN3tuN4lO/lpzNfA\nLpa9I5l6kvQJ9ezKEywBWrWhPzTihF1n5d/O1fGwoBZV7mXPxkH9ChMU4Uv43IDL3EOtSRaUdORD\nlpzvyQuKqPdSKGUdfgzQ11xvJO+D5GvchziPUUJZOy38ucwxR74NoS/Sb/Ccpm/oV0Ce2p2BuKwR\nZQGp1N1AMea8dzIDWNtzUqD5jf6PQYxRrXPxvc7tPAC6hheheW+eZcpstOP2PfxfSTcZ7dozKJq6\nh5ne335i02IRT0GAE9BVizOM21UPXGj45AvxwH5fJXznqRPW/jaEiSb505uwhORbI3Ui86tkuMy0\nYZGneCasFefH2OlMOFwujcAwMQxDX0Ze33YXE3wbP2aj1FRHMHrN4HkibQ/PfXVijLXFbENtHXVY\n7eX1O86JrKMoLSntoob/Dsq9YqgProuN3K7KgOQDApczIbQnqTNs/Wn4hoUwKHkwjOe2i1WRyMgn\n7/yTxECMI3zdAJYgmwIk1FTyXibmeGATAy4qTtHncUW6MbQD18fBdQxqBl+mb1Q8wQFPjhicUM7X\nhI+o72mPixHogW5E/cznVrb4c4mGxu0FKaSQQgoppJBCCimkkEIKKaSQQgr9L+gvRahIADSSF61j\niVZwvxeImmJiTl3tGdvRiASPtsaKhzoTFpds4PvnvE/I69oTRJFozS40CWaYqnIHYcvtRfO+1IuM\nLraddgFnigmfNr/LLFveIEadTD8HwSaimtZ2zCz4lfBzBF9klNMRYsLVh/h+o7kG8q/e+mBFgyWH\nTYcrJ+OgimdUT5dXA/2PbOD23SRGi4P6EX1SYoyAKpZ8XN1yBQBgTDkjxI41vMadIKP9AEaQdx5l\nFtYQyXtLB5gV8NMB9kuiSZyVUUjVbcxym1U+dLEwM3ByN2GzvjjKsrwtZauvAKIHcWRkzpGY+mFE\nAyevW42KAgvMPdnMUP1FHoy9RHa0iJDHmmaM2GpsEoJO01TcN2MbAGDFApbshJ0QKBS1CkUvMQps\nv0Tkgf5nZhZ8BJrAWCzD3YOlCM4aZtnqnuf1uq0GVPOxmP0RR9rd+cFuPmsdURa2JjL8txHJptYB\nAqzUaElXAES9LiHjHtos3f5QwCSatZmYJYs5TCYVSkYYKqgP+57h6PLxA8YCAKLfpr5ctTWHqz11\n79YWzHxv28sIf/O1/Hvusz6ocmlHY3ZTlp0Xcuzcmu29oUqgvvvtYnYo5w5m2/wuMLUXmO5FkZky\n9LVRMnEAEOVXhbc7b8DztrsBALV5evx8lLDYpsnc+8pEY2eV1oeog5RX1u2EERmKuT9prXytzG0C\nbxuuA4+AvjZdyevzBoqRkAkS5kxdAgCY9yT3VVsWddJtkm+UDYQc4IZcNZN2wnOM6E61BvBOY1bd\nIEuQtjRuZQwLrMZjd2zFolSWJ4Z964ftZ4jScg+mjbs1njq1c0kPTB3JEdePvbYWADDnx3EAAEmA\nC1o2L0CxkzpUlUj9fvwwx9oHCDRzYGAdnF7a5WbrqYtpU5l1q2nrws8Z3E8dIhM3dC6bmO44xGzf\nPWcT0WE2G7+HiO9xYd5NMqKBk09NCHmzrmxe73s9DF49+WdtTaUoyia3QgtleA3UJxcTqyjpJkoQ\n9vL3ivY+NGub/x/P2HqhNQDAKNAOpS844TrIG+jEfRxB3EMrerqgqqSe+53l5/DPE+UOxdTpzMku\n+FuYRffTueG73Lh9VJ+GY4yDxJhrlVeDygDuh2bBs5rmgocFKliPE36p6UkUy5HsBACAlMFr4nrk\noTCA8vD4CbRRFn2blKVElmX7tYZVdB2O/pk2O/1hvrdN+2xczOf5xBNGO6mr5LpxC3S2JtwO3yk+\nL6CxT0wQJFs1cB8IQfPhWQCA6oVxcFUTzfDdY0QpRHxEfSvtJMHFP6HyZQrC/DFt45k8nv98PhVk\nf8H/DPJ/52xWL1RO4TUxoVU4eJGOaIRTjLmvFmikc35Ae/o3LT64PsCB6yJ2J+2vT6+GVZTHOk6G\n3yiBbqwkawBXoIzC3tS7ZmvroPmJKOrd75LP7mjRImJuE2h7UWazHua++PZJthoI/4VyVnl8sHcR\nyFobr2u+hrIoeob+pDsnAKe/YJVD6ASiYLITeG5s9WwFrjxOhGCrt2mXS7+kXZVEaaX/WT1qWAUP\nzZ+cQaMgVBRSSCGFFFJIIYUUUkghhRRSSCGF/iT9pQiVmmo/bP+lK5xRrMF/4sedOFDBmsKqUkaQ\ndKKeMWd4MNZ9wsx0zGRmUd548TsAwOupowAA76Ssx9xsZsr9R2QBAI7vJDKl1zTWUI0OOY0nI9gM\n7pGmTwEA3CkMO/n3qYLudWZLe05hvfMv+1nnnD6ZEbGw2FL4rWYEuypZB9mhNG8DAO15RtO37FuK\nxN0PAAA0x4giUYnEs7pjDcIDGLntvZ+12ts/YnZ85PFHAQDSJTM0IoyrdjC+92yb3wAAb5dTzm1b\n5uD8VSJLjJmMFjsKmCVX21RYuod9ObqOIzrp7AbWNYYPFaiUk9G4msP1Zapt5DWN/5UkwCZqQnNm\ntEXTW1nne71Z6UYzM3EahxqOUMrGIZprVHSknkbey1rvS2fjEaLhz50GUff2prJmv01TRoJL6vzh\nq2Vazv8y5ViiYxZW7uxAsBixO+XpLQCAT7ZwZK++LTO8xh0BkEfyGe7DIZAbeTjYZVEjZ0Qg3KJ3\nxit3rMObp9i1t1McM6xpeeytoLEBwZeomN2WEiXmCqIMY5+nvF0VAegax/TYdWRKcEvWLHdYmA4A\nSD/bGRqnGGl3C3UwSzT71tglRCyhDHPGM5urLaSc75/wKwDg+6vdINeIOvbLhvphRAOn/NogPL93\nHNRV3I7bzzyD3bvI06JdtHt+Q4gGkVKDoanj/jVrKJsyf3KW+6R5K3Vr4OTj+DmN6AS9ljKOeYMo\nvsKt/Ls92YltVWywUtJJZMAL/917wS0QMYUDeH3IJjGeV7S96Xn/KRwuSAAAVGcFwuNo3I0US+oC\n8MnRIYjeQv8ge5QP0FDf1A7yd8Ne9k0YMuk08kTDzLn3EJnSdHQWAMDrU914TavinmWPpX4b0qk3\nXqN4qE+FXWfp75j78RkfDeZ4+4WPjIXHj/qVPYb765Vq0VcniJ/r5W5bsPwZ9qUrb93Ix3teJwmQ\ntTJ8rxMqcu1hQFVAvWiZQORB7pYEAID9tmqYfqMcuw4WfRqiyWNJxWva+tegW1AWAGD5Oo6m9hPo\nL49AExp0blgGMJOak0ofyhLPPe/euEvY/QGRThWtBfpQ9NUPP06Zhe2QEDKVqGD36xFQFzbujVGS\nAZVbQuBVZq5zn/DAV0LbGLGInYWrvyZKyO7VIW4bzyOVnURPqCrqjSGJvutdkaewsAf3Olu2QMjW\nksdZi5ntjgvORsYJprUru1GGoacpr0vRkdDp+QyfGLntEm5oUhLlbtS4kbuHvQczRwtbuuOmWdGg\nSeXvgaFfGQp/SgAAtHzyMk5vp71zCKTRgmVsjD/96+k3EMvVtTSQ5XeQj0bRmH32lLXIc3EfW57P\nnilZ91BGOtEw2qBxI3Mk79nnVzbbh4F7oCtQDdUVnntafMOzRt3b9JOybuffF4xfjJkr2PA2KN2H\nAkc9MKIBk87kQlyPPNQspR9T9qIDdid5ZdpOnQq5h77q1SZhaP4l9eTtZvT9NVoKVV9JWRZPdaDJ\nD6KxNC9H5hjqq7ec7wm8okL7x9jE9syX9HEswp+59EYTSBoaYGk55aoSSJf4jjynZGmiYCilfrsC\n/1w/qsZteRVSSCGFFFJIIYUUUkghhRRSSCGF/hf0l6aV9GYXEvtnIu1IAgBg6TfD4OrFeqq2kYzU\nntnPrHZ0n3wUVjL6f2UHUSzPu/l6xz37AQCP/joVYUcZE7JFMAJlEOiIg7uZidMM8mFkMKNVpV0Z\nhewisrfHLzdDaDJZsHU+kSnDnjoAAFjzC2vrBnZPxwE7M/U+rQS5sQMcZEDySDCJrE2rz6ejywh2\nxS5vwshj1klGI4OMThRVsbBRe4lRxenvPggAeHf9BgDAc+mTcDhNzLgOZDRyTSFTMJKTsr14MuFG\nR3tPG2YNLEYh6N+CoRcTZ9IWM3rt5lAF5JYF/vsz14pa8+FEO12ec5N8aOikkqEyeuBnYgjdlK1H\n3xC2xf/6LNe+2o/yaPFEGvanst7R6mW0P+gsM7EFUdRRVZgDOg3f3z2APD5znD1ZKjdRIOqHS6G6\nzLDy9WlZLWOZWUs9EQ+PH5Vry0T2ITAO5e9uG7NClW09CNnIDIOjre/GJJrGSip/D0y9SxG8mDx5\nP28c1O2pH8cvM9slWkRh3D17sM42AABg4GAz6CtEBieZwnBX65FeycxsQBJ7ZticRJioITqgO1VI\n7E60w5WTzMil9VkCAOi9eTo6v80MoP4xZgBrEimk7dtoXw1xBoTmU3fzBjbuaRTXSWOVEL5Pgwqq\nC05/0QHPP78eADDnFzE+tUZMFmleg5JOtGubJ5Gn3qdE/zE7ZbQ7twVU2czSaVKY6T7/PfdDZ7IY\nh2xy4ddU2ktfc2ZsVG7RQ8Usw5PAmvLAQ7xPpRh1bcrjgtq/qtMNHfbrXw2VrnH3bYBPgmRX487X\nia5ccGAIVGJ8p09MHhSgBVyrDUXRfcxsOzoRbZS5OwEA4FckJhb0dOHJbhx5vsItJoLEiv9dImLW\nl23B+8N+BAC84LmHr+vYD8c3WkZye6LNWo6joDK/YP8B2Ua9//a10bA+zHp/W62CFgO4L/kVqHB1\nEnnU/BsvikXNvu1D9mJQC3dlXOIJLL1EdFjFFNpgfWe+JjyRBgA4/VtL5OXSFmuFO+IMEsiUct63\n7Hw4Xh+9GgDwSs4YAEDUdMpl9+IWKOpH3UvYwFfVbBrw0gL6WY5QGZqFtMUFkz1wZjZuJ1XWynCH\nu3FNAAyaLtRCeoXnC8dm9jIx7uZ6bzI4DyVF5ONTSb8AAH6cNxwAkDGVe9eSd29HXUdxc4PoTRVI\nWYxP5Nli/aq+UHXg3lsWQ3scvZzXh2wzoKQ3f275EdGg1ybxc1TE0q7XOXRwthQjuIOU3mIAoMn2\nIvyhGlx7mPbuWHY8AsW5o88IIkRmfiwmDQ4pR1UVzx+GC3w1UhwwD6ePOcY/G7fP4H464Q02OVpy\nkugvpPOa/JP+eDuUvRjHvkFU7YLTRLN4/HxI+on2+pfm3E99LJCAQSBm3s8aDgNBvahMkeDdefN8\naMjkqdKhdEMsEqZxYlb6tuZQd6VPUi3sqPUg98Lxtx/EsOWc9DrjM8p14gPcT5c/xD0wPrAK+ZPF\nKORiniW0FTyLeH2UgdcA7DpIZ0ofw7/Zo7gXB57QwSH6VNmWcTyluxl93LipRNYXxFgQspd+j76M\nunjtD35fBaGikEIKKaSQQgoppJBCCimkkEIKKfQn6S9FqPh8EqwuPbRWRo1MBT7UZTHKdLyc4apn\nR/8MAFi08HY4OzI6tGXaRwCAmVPYhyNuCkOAI7qdwW+VnIbgx/JWVHXiNc1WMJJ5vn0kjv3ArvZS\nPKPK+Z8y255Q40Xxg0TIuK8RSbExk5GtkHO8vnaoAUUiiDl76Ca8/11VfbCi4ZIK8Jp8qEphLK7Z\n2hpE30meHD2XyPeEMiNWfSoUvubMdM6aSERKkyl87yfZQwAA3fpfxqET7PPQ5CDXRflhZluCRbiv\nvLcbUg0j/AGiHtJl5rrxBgDWWF6nq+LrfWM4W/6X9wYAAKrGWNE6nFmdAaHMHP1ys3xo4BThV4tZ\nnXdg4VLW0DuCgVANdcFykNmbKpGVPrK7NRDKCG+KkZmeDYP5XpfoiRIVWoVakf35QjcaABC9l/1O\nvtqyGAAw6qPnEDWKfW2qf2S2r/oTIesACW+++g0AIOcxIsJK3ES/LNk+AAAQGlcFexOuAznXXC98\naMjkcWhQkRoC/X2URcBaM4piKLvQOOpZuYtNL1atHgDLsGIAwGNNmZ15bwX7N5h1RClJRg+03zPD\n6g0UExJMfN1wiqglnV5GTTyf4TPSRrb7bAYAwNVOxr7PugMAJn/HPjircok2yzrLuvKYTgVwfsVM\nuTuwkbfAF6QKccNvcgEqBaLOWm3CZ5/fCQDwtqctHZvCTOiv3/eEyku+x36RBQDI2sb9rWgUba3G\nqUUgQYNwiHtasnif4DFcAyXbYqAXidDgS0SoeMQEhLzBElQCTlHVhW+K2EVXoZbJJDiDZchaMaHi\nckCj7y2m0nphbGLF56f7AwACLmvgX0DdqWhJ3jTpyxpt+xdRGPjicQDAg6FE285YPRMAUNaafF47\nYCFeuHYXACDoHWbLtJnMtLbdQGfnxKp2eHHLvQCAlC+Y+c4aRz3TFUjIiaPu2+cxAxe0nXrrEZlB\n/0dzYV9Lgfo6u+uFDw2dfH4+2DvbELKbPK9K1CBmB21p6qP0OcZ1PwQAWLxvAN66m8iSVX27AgA8\n31LmI0OZaT3ZMhamE+R7+D72QSrtSxnpa0QNv1uNefNoi9X9qcM5ExMAALr1MvyFLS7mI7At6QcA\nwKDTswEAfoUSqprzPXEbVKho5C4qPBI0ZVoYSrnucwfJ8JwTk16M3HNiB1CHrEuj4RUTPZrr6COW\ndKTs4wRqfs8Htoom6wAAIABJREFUGzDgApEN9uX0cUK20I/8eWJfAIClxAf/vWKCUDPKO3cC7aqv\nTkJYLBGf2Xfz+jtHEQl/vILo3bLcQMDI9RCxjT5u1k0zomGTO16DgnmBSA4kuuHapuaoGcQpO0cL\nKbRW47nR9Qi8hmt2Qg92+PE8YdhIHzE/l/5kRUsvCnuJ6T6v0Z/xb0F7a4sRPVTKJXy7lzZcFcy9\nT3+Z62H8uD1YouF1xsu8zh3APTCqD/3aWqce1u7UYe0VI9DIQbheg4zqlh6cvcR1rmlrg2Ur/fq4\nQzwfWOeRzwPMl/FWJs8j/3qUvcBmb+f+FpBGub3+9DI8+S/6m3a2R4HKTb0LjSLypdweDClE9Ekp\no+xaLOfvpR012DXtAwDAIOk5AIBL+KFnS4hY0Rw1wyF6p1SkiFFs+//Y9/1rAyrlWjiWNYEviR82\nfHomgibyI5QNpIL8tI4YqlveO4wduVSMe+c9CwAYu4AH5R3lhCrnfNkC3775OQDg+SscAVqVJRrI\nPs1dxazxoP3E0wCAnemEcpnTKUDrHDs0v9G5nzCF916XRee0ZBQFsO1IewSfp8DMwx1QSY1bQ7S1\nMmJ2ysi9nQ6Y37xirD9PPKShmLK0XBWlVU+ewvYMynDpa1SUol7kpc+fm0fZpRiMuo/jsfddosdQ\nN5BGM+4LKlHIhFKkXyAs0zyOm5zhGSqKZ34dMvJoSNffswAAMGop10vSI2yyWnqsKWzzGJ1psXxv\n/TCigVNxjQVzd4xA3AUR/IrXYu73PMQ5unPtx60iz+76aBu2lrCEY9FLdPITnmDZXOlyGso7Zp3F\n5tGUe/U6Gqa052iMxj9PebiaA9ln+T+kUI969OfI3UPfdMLjm0RzY7so9Ynl55h0C63Z2aoYqEQD\n44+7fIVRi8rqhRcNltQyvP5eGH7lBlXaWYZGjM4ty2eZVOhJyrDTo6dxvIg2dkMJ9dVYQhmcOcIA\ns8YN3PsqAyGf/8SmYFoBm9V0oUPotBqQL2zs2J7U2+t2WvdbMLx63nP+CTZgVJXRqVXFESpbcDQK\n6paUb/gRyjL75jnRsKlIA3leOALDaT9VXt+NchrjTspzc2EPAMCj07bgRHUCAODCxwz+q0QcO+AI\nbaLKI98YQR5wjfIovp9Bs4BlPFjUdfeiV0ceCjI/5r5Y14RrxZIKhDJ+g+oXhZMygpDo3X0+AwDc\n8dZsBF3mPad9sxGv/FBx02xoyKSqUsN/YwAsLvLbObEMBUXUQcs5ylCez4alNQ/XYHcuBbRvKfe8\nWe/xYD7nPMsN5uSNgH0hbeXMpSzrWfACy3oKS+jb1LRzIuQQ9at4Lp8RpGXQRfVVGBw6LqLeMQQs\n+7cSTqXo5nckqylMKupidAzl19gntkoOFTSpfpCFn1fTArAOJW+lIvJq63Jm2NY88TEmfctBB9cb\npDsHksf/OsGG+r4yPZzTuU8VnGcgxS+Z8isr5IEvIBUwFdIWVojmzuEneZ/KJB1if2ECMXU25fZS\nHhuPa2xifOjAMli+Y/DM+VhFox+bDI0MT5gLyT25s5TNaQqbsK3VLShL1UYxKvnJYjiKyLsPhrPc\nyvoaD8QBX1JerVOmw9acZwYt3SCUdqf+mnnWR1FPQNOB/k6TjtTBlMe4hlJnNkH5FR7q/bpR9gdf\noz1v/RoDb5mWEJiO8XrDFPq4+P5mGdGwyeuVUGs14mwB/RtdNyuMh0XJuIf70XkzZXdBToG9NeWW\n+DnXv/lDNtKvPEBZDft+NlSJ4mzRh5aubCsHWASfoy5VJ8mwxIkhCCuZjHCZKcet+a0QuU+MsH9j\nFQDgyxd57mw9mAG6nw91QugpvqfJA9dQvNJZP8xoyKQCzFeof9bmEso7UT62COqEXZSA6xK9uHqF\nAcf5CxlI8ZtAB1RzlmvgufSxcI2lLyqL5sPuMtGGQCQeKsYBLebznoOX7gMArKwcCgCI3FGCiWlP\nAgBMTSlXRyQ/T4CBssoPk2HOFWWZpX/6qyqkkEIKKaSQQgoppJBCCimkkEIKKfRn6C9FqEghbugm\nFcN9mZHfs1djEfAZI4bqbXyP08II8pbMVrCVMAqJVoQib5rL5kAVbRk92vPuR7j1ODtPRc0XHSon\nCahOCSNafj+rcWoaszg9mhKxAIJakH4sBYZejEYuOd8TAKC7xChX/EBCuDLLolDRkdmDt8+NQKEt\n96b50JDJL9KODq+cRtFVooQGh6ai4GtiiMsEn6yiKdcvp9phcg/CYzdMJD6raxgh52d2Mqs98+F1\n2FrGBk8PPMHs+LzDtwAArt4rMqYb4mBiEBIRXVjecHQ611DIyiCYBCR2wn4iISLHM8J/OZ/oI32z\nGqRPZfR/1qoHxDd55mZZ0aBJ0nlhiKrDsoVfAgBu/fY5JA9kuqXgOzbRa/cWR4l/cakfnMXkn6k5\n9TM/h7xtNTkLAPBdeg/80vlrAMBIf0LpmkczvKvbxwhwm6eqYfdSTw8fZVb87ByOh1WFAKZ8ynvq\nVK6Dag+fuW7JAABATZIH/xrI0rHbTj2EDNvi+mBFgyWVU4Jflha4XSB1akwID6E9s26jfPxKaTt3\n7ekAn4628WoC+dx9KrNj+3ZQN4M7leDLVEJarzf8fe/hbwEAM3ZPAgCYrmrhbE+0yaYtzLJ5/Hlf\nuaUXEaJsT6Pnc9U1hD87LLyhWg8k9GR2KKsNy4saeybO7S+hsLcGEV2Z2SzfEwlbM6ILTBnkmyuO\n2ZPPzg6A8Qz1wjqEWRijhUiRaiuNpKZAB08AbfGAGUQR7fqSsrKHivLIMBvSFtGGV/Si/HwBvF+L\nbzwo7iqgrr/xVR3M9/T3PgEAkDv4II3mOnhpxzgU1nxSL7xoqBQQYcXgpw+i3E2fZf+mjghkIg3W\nePJOa/23u1WXR3SC3Jp6opNoIzUavqb/lASnaFL86poJfO/d5LenUvhFTjWstzCD57PSb5nYlmi+\nNYahMH9KH+jQY00BAK6LRMz8MJGyOnTgSdia8LNNiiFS8PDNMOEfQJbAOowYfQSbdrJ0UYq1wVVK\nHQg+S7vpV0aZjd3zGPRCpGoxHjVxkUDethPNnDu7YdHzn7YS6t7ZSSsBAK1F40WPH1At9tXQ/XxG\nv3lE0q5bMgCeQN7L/xz1+6BMf0sK5bOCvw9GYU/e2+JVQW7cIGqYDQ4MaHkFZg353vyd41izj/bv\nOhKhoB8FF+xTQVPI84FmEc8iJ5otBwD0NjwMAJjZajcynURBdzMR7fXsTqLFQkfynFBXHAKDaPBf\neJp7b/6b/D35zVJUdeL15XXUwZLJ1FtVFf1YbaofnMLGVm6Jqh9GNHTyqOAr00PjpMzkahNs3WgD\nwzbRr3CZqS/m/sVwVNKmhszlGe1oZgIAwGehnnRsk4kzWSxxPHiZsE4plPukOYvPeOL2LZi/exgA\noGYk99wW0SwFK/8+DkGXaNTD1GwaXdSdz79cTZnD4kazh7IAAGd/TYGjRl8PjGi4pNL6YAqvw6ge\n9DV/ONsVURFEaRV6qBPNl1MGj+Y9AgibVtyFfI36lntd7lC+x70lEgMn06e5Wsvr0woSAACuQOqx\nHOzCtdkCFfogB1x4e4g1ZNCitCNlMva+PQCAjV+yxMsTz2cGtCpHpY0INmdTgTD6g0cNBaGikEIK\nKaSQQgoppJBCCimkkEIKKfQn6S9FqLidWuRdDceYvowwbV/ZAzXNGdMJHM7In0vNSFS8yYqSbcyw\ntJp6EQBwvIhIBjVL5TDkyGNwFTF7UJ0oYkMaZvYsZxitsoXLqBEjJ08fZL1c8nDW1pkzVKgFo5o+\nHZ/raMGIVGEVny2bPAg+xixhh6mZKNc17pq4GqcBv2a0hKecWZO5x29B+ARmyL1ZzDjbxHjHh3ru\nw6JDjP4Fn2LEMK+CfJVaizFjG8dAjqNA03/iyGwkMQMEA2Vi61UHTzkj0mUvJwAA/Hpy6ZZ3d8N0\nlfKpGsT7eDewT4BZNG+0xhshmXgvXavqeuBCwycJgCTJuGUpG9vJBhnuydQZ72es2Q7XEQ1k2u6P\nFvcR3VW9jX04goZQXysdXAd1OQF4I5KR/ahhRCBUOymzoCrWl+7Z2w5de7OJWOIqZhqu3C/Gderc\naN+cmYUf5/A+xim8zlBGOT79yAa8uZN1ztfGfIVupkbet8HsQUC/YlTXUQa6VCNUFylDiS2HkCdG\npoYEV0KnZvQ/P5PR9x1FLAj3FzohyxJebUN00JeLWRv8rHMqACBAZNunPLQNK+exz5VoZ4OqFOqy\nRwLCHskCAFTvIMrpej+lIjG+zhfuRN4OrqGut3P04dWbY0ODJ7WfB5aOZfDTEiHiKJChcdCm6QYL\n9JGLvxu2BsD/biLwTMuYFSvuJRpDN+N7S4oioKvgfvjTOTZtl7vSplrO8z76w2aE76K+WWOZtWvd\nOQsAcLlfElqPZH+VrEW0ybUtKEf/01xrYSPybvRDim9XiHJD425qWlHtjzXbe8Mo+mPYE1xQuamL\nYadov4pGUL5JATXwtmKGOiOfWbaXj7PpZdtYyvZsOz+YArmf1ZVRvqt6ME326jXawNzKQDzeikiG\nr75hj7Kv1czIuUc4ERZC+93Cn/p9xsy99+5NbIDboftVpG1lf4Hvlg0T32T7TfOiIVNdoR+Ov9UF\nFjFas1JjhF/xfyJT7A9RxgE7QxE8QjQadlOvyst5oeVOyrH2VBT6dqOFWx5MPZtZwL45sb9SLvYo\nE0qncD8MfZ3PWLVmAABAP6AC+Qb6VfYm1MHgI1xXFT24nsra6xCQzL1wb8dl6O9XXh+saLBkrTbi\n6Oa2cAr0QdBFCWjPn8t7kr9Ji8m7zAQTknpmAQDOX+Wm2eXULACArKbeLtT0Q20N7d620+yfs37m\nfADAo6+wH0PS6Qp4gqinskDGyKI/UfqD4Zg5UjRpf416VuHguaO6Gz+Hx1+GrlIgradxVOxL82+e\nFw2ZDEYXUtrlwDuQupT9Zk+ErCe6oEj0uNQbue90D8/GCYl+xeHLRHDFbqbeFvTl66jws7hQwB4d\nhrOiwbTonblrM+3mgk0jEH2ca8WvgP5SbSzXhanOg5hF3DOfXP4QAMArEBX5lUQe3d3uFHJt7Osy\nddyvWLC2pn6Y0UAp2liFd9uuxw8lRIjFr1SjJo6orNseJgK+2SAi2Tc9ORgl06k7nlLyM+cu8rdl\nAs8C16ri8es1Imt9Ykzy+KFs8PxLFhtESyoPwtdSX2sT/lOnvurQD2v6fwwAmPw1+1/ZW4kxzALh\n5PWqgBbce+9pfRIA8P4f/L4KQkUhhRRSSCGFFFJIIYUUUkghhRRS6E/SX4pQgSRDNnixYV83AECT\nIUWw5rDTb52dUffgzYzypvYIQpgAgxzfSmRKjxHnAQAHd/F3025/JNzNbHhwZ0b4rYVEJ7j9eb/O\no89j12X264gUo9Iy1jMrc/dDu7Dyx0EAALWDsSVPL94Hoqtw3xEXcCKQkc87Q0/ggKauPjjRYEmW\nAa9HDUmMqprbazVmbZ0IADBEknfNnmKGZOljg5C8gVmykq6M/vV8ieikcB0jt1//PBShQXxPWaTI\ntFcSzWLpwLS4xeDAwNZXAACPjebEprFTWMufHaKFryvv9VDyUQDA/ijWR+ZWsUu3+kwQlo3/AgAQ\nIeBNifXAi4ZMCX7lWNbpO3wT1w8AsH1vB6S+TeRC7Bc0C9tMRBeNf307thRS5255l12zvznA6yIS\nmBlrutGNfXYW/XvNAroghg303c/eLKnHvEirYAavYgqRKXFifnXRfS7UeaizpvuZkXB7uQ6uj8X+\n9MpATOnDHgFvlLZCgedPtuD+B5IsSwhbQpvZ9+39WO3HHiixO2g8I/dwvdvig6CrESiCSUI+Ipxu\nLWOdqvGgBYtyOOnJ/Rx1WDpIFIQtitm6pVe7I+ge2lG3jzeImccMak28Fql29muAGKlcMoKfI2Qf\n5a29swJFLsrV5Wvco3avk9enQlWtEZXVlMOUZ3ZjawHRlMViMkjkIVHrPTMb2dsTAADukdyLxomR\nygfmMAvkHybBHkH++wfSJptWMeNT3JNKGX5Uwvx9nB4zM4MjW6tfZCbONsmNEhvttfoe1o9PiiKy\nbE0RbUL+oWiYREb1yVE78ZyucWfioPNBjrXDdY57zoChFxDdhUiGvW8wq910KfkVP6cCh1Zz0paf\nUIHnHuCUn+ujzGP7FCI3k7ZSa6EOPXR2MgDgi3YrAACTch/CmjyOJf/gMY6cf2zHFACAJsCFKtFX\nxS7QTZHJlKVX6G21ywhHONeVuom9XtjQ0Kl5bDFWfToPA488BgCQHVo43SIr3p3C0hwW9fXxPrir\nqSc4Q3+xzWQiu05kcPqdpJWxKVdM4xK9IH45QJnhacr19pancOALolYKBvM9wanU0xJTEJoIv7V8\nD7PrHooV5vP8XOr+FbA7uXe+V9YVRZ7d9cCJBkxGH+QOtfCJ/hXWoU5MSGKPoD1vUxfVaUQaSKpY\nXN2XwOsE2iCuDfmdX06bGfm2Gs7nuHd2Gk90+51biPKS+wpke0go+k3mKPTuZvZZeWf5eADAHUMO\n4+ulIwEAr86h7s67yj6BZVVESsR2LID7K+61ZW7/+uBCgyePT4XSOn9U/sA+ewZDDQot1DdDGpUg\nOJXrftu4lugUzX42EOhcfQXlJ4v2Ju+tvgvNxHSfnJ6U2y/5ROkGvUyZ9zBY4etP+3jtU/b5K+1A\nnfzwrhV49Wva4NgD3HuzuAwQvJJ79477k2BeyOcuX7QTq683V2qklFcegheW3Q9PayIyx8w5jl35\nRL2ee5Vy3d2a+9PcLxdh9mdE/jjEGGujWUzeqSZP2wxMR8UbtK3ZI0U/xq8YT3DPIOLPV2FEaXvR\nD6Uj/dgVS6hv3e5MxUMfEFXWdDxR9xmltOcOUXUBjQ9SHc9APfz/HH5aQagopJBCCimkkEIKKaSQ\nQgoppJBCCv1JkuS/sCW4PjZWjn7qaahZNojgizLiHyfy4Oh5Ygbifub/codLaJ7CTPXwCPZQWTmX\ntfvlAxi1Ml0wwBHCz+/1F5kWK2NECV0ZrczfEwt9F2bRD3ZeBgBou5ZhRV2lCtcTa9eDwlrx+/WM\nrBxrh+4iM8BNl2ThUPGPqHYVSzfNjAZK8W3M8vNrO2Pu5lEAAH2FdCO79q89zG5LTspA1vuwcMhS\nAMBzF/g/l4uRP2cFo4GmLM2N7uZNN4peKvcz8tg+idHktJJw3Nb8AgBg43YxWSSSa8BwxYA7xrKG\nbvvnrIPUV3MtFPYRne/TVdCPYHbuekYg895XTsqy3KUeWNIgSd80Rm7y+gzoTVRGt0uDwP1EEehq\nKQ9bGOUYdNWNVz/ltJdH1rHzvTeAtciSnrzWFOmQsInZ8MJejNar+hBhVJvD7F3IaRXKO4nIc5SY\nTiGypcad/qhqyedG7edrwGVmeKet3woAmL11AhJXcY3UvmrFhSeWou5KYaPVRWNkrJwwbRbcYspO\n6FkZRf34szGP2VRzDvld1gkwZ5LX+ir+TQwWgc7K3/P7qxB8gezU1vFv17voVw4R/YmcasSv5t9y\nbuPrXT2JOtuxuCeq2jOTZyikDkftp55eu4/PUml8UOdynQUwkYfTi55p1LoYkBwhd/9yAq4WEpEg\nSQAKmV1tto58zyIgD7ozJgwfz1ksG7fRFrrDqItBEdy8qmv8YPJnZkzaw3puRyjXxfVJQlXbIzFm\nMvtv/JTBTJE9i9m/5MUViPiWGbvCx4jOLOtEHbbcy54R2cUhQAHl6NPJKPzwYzhzchuvLiZGyc3m\nPgRrMfcXyeC9MfGloA/9B0m4Ws5OVkQtpXyrm4r+Xx2pN8HHuT9W93fA7C96qFyiDLW1ZO87D9CP\nmXVwPAz+1C93BmW35G6OMHzqrcehvpsIvgqBfGrxAv2g6q7sfVPYW0KLZUSHpk/h9VlPPqvo4pcT\ncOUa0SBNdqvh1ZHvZZ1pEyMO83drtAqxI7IAAM/E/QoAeOggEUKyS0yp2K5BaSe+3x1KGcf/xN9r\nHqO+9ojMxm/pzIarRA/BuwSiYltuSzgEwihiEfWtKpFZeWOZ2EvL3Cjow/UUdtaDM7s/gbUyr9Hq\nol+LSDn546kwLqHehM7Mgk5NG5n/Cc8Zxd3JntgdXhT0ps417U1/89oR2jxPDHVLm6OHxiZ6gAlM\n/8Sx7L3hEOPwds3pjcLBYlJXFd90vbfDyj290bpjFgAgex17i0XtEU3JJN63pJsFNQOo7x4Hr895\n4IVGrYuJbf3kjza0wOvzqFPNJqTjviZHAAAfvUhUfHUz+jm6ahkVXShjrUA1BOyk3bMPo56FLDdh\n6JtEWO8tZZWC6mWukZyh/uI+QG1n7p3vdl8PAHjx4F0AAP9UHa4XKNRFy+L9lN/1/fWTMd/hiW38\nvH55amR+Nw/2wsa7L163p9d7LcprQ2GNE0jNXyiXa3dy7wk7I6NgEG1aUgue/ct/YN8pewSvUbkB\nbV+iTry7WN0SdIV2NW8g9SaqXRFcy4ns7fE0+7ScKSdsyfC8CS+s5ZS1Jz99lH+roOz6Pc219fPG\nnnDE8J6SjjqdPfmlP6SLf+3YZB/Hy1k6ieZ5cf5wrORG0uwOBkAGv0vI5LbX+qNDT/7t8+1DAQAq\nvhUPdjwIAFju1w2+fCpNVCKdh/JDxHe1DqRDWJMdA3dXMmxlLQ1lxjiOim1/7F6oVRSgfh1h66X9\nyMhxHSmIoy91Rc49VLCr80LhfOGvrZL6u1FFjgVrZgyDRgQr6pJcWPQiDU73ZwmHfCdmEwBgzKfP\nYX0nNkVMCqF8rkPJc0VApdnwa7gkxhtnz6QshiSwWWV6NQ8Y0Z9r4ZvL500buQPAvzeyFYX9kWKk\n8u26k89Qi+abKgHZjFjiRmYT3mv6HXR8ZtcDLxo8yRLcTq5nn0sNUzH55nuU+jlFjNJcdLk33rjK\nAJrchLqwvg91aFcdG0QtODYIJV2oi9MeEI1N144AABhF6Z4tEohP4YHuzijee8knfE9VKxmmPDqh\ntffR2Qiw0Nl/J3U4ACD522rYo7nxpQSVIEPduBth+nQybHGeGw2f3SYJKV+Qd2nT6Ch4jdQbtQNo\nPp7B65OXWJaz5VaOTx3xGyGQgadVKO1BpySxBe1n5W5uaL4KOuyGCBu6v8vSyz0RlOExJ+Vw4PZm\nSHyH+p1xNz9Tz7kMtvie5xjSop5GRPSivta0ESMFF908LxoyeXwqlNn84HNz/Uf+qr1xiJu+9CcA\nwKsXqH9BqR6cqaRzcD0g5RYNTmvSKXNZJ8N7UTgwfei06I4wIKJ/myUpbd67jGVHCX/vmJIFACjx\no0PfdWUa1qxmaY96CJ/Rfxybs+1fwXKFiAIfCgbRXkiuRusv3iCfUw17RgCkcAaoW75egbI+PJTP\nmLIRANDfj/vjhA+fRWFv+iTqFMonOZh6W3KKPoo604DaZnxPq16EJl/fJ19cTth50oBsWF3UoQdH\ns5ns54WDAQC2SAm2PK6H5K8p144bswAAP+xjQCXwsgT3h7SxHfXFAICsm+ZEwyYpVw3VLDOkB0VZ\nVFMV9IzrQ3W9LHwCHfouoYUYHcIS5Lcy2BRYKhejO/2oG1WJKhhTKFuz8EsqUwgxl/fwQHDYEQIp\nRgQ8O3N/3JzFElvDZgsee4b76dftWTYSeguDmvknKEdVcxfcLvpOdV3q4DvvrRdeNFTSFKgQ9qYe\nadPI057mEpyvIq+KxfhUdYxoERBpgiuCe97Vk9Q9bwh/N6YxgGWPcyMukXK5mk6dvl7yrCsX5au3\numEMoG9kl2h7azy8PvSUhAtalinEpXOvzHqFflfsxyIgUCsjJozrZH4iE5Sd64EXDZmK8kLw4XOT\nUHkr1/OZo4m4EEf+B2kpR6kXeWb6zoxlt3wKALhtOxPm2177CADQVfg33gl2/HiVXDXqKAd9DGUV\nPyAbAFC7MAaSTLm9puWee705eMCgCnzXkiO1Z9zLkee212g/h0XxzPLR4/ehy6ssE5lw61E8t7lx\nN4h21ulw7UQsdFWUl7OVjKDL1Mu0R0SJjVckHgYBnwz+HgAw6zhLX+WB1Kn72tKPXLmlH5pbaJA7\nPkA/9PD9LJ81FfC8V1kcCedt9InOvMb/zfqYpXZpyyMxbROTwiFVIpH/KGMOP2/oCQDQ2IBX+mwG\nALy79Y4/9X2Vkh+FFFJIIYUUUkghhRRSSCGFFFJIoT9Jf3FTWkBWA84dRAuEF/lgZQ9ZFO5m1u07\nM/+Q/FQWOpuyAADH2zC6615EGM+qImZhLKU+mLMYwcqczgycSmTrgjSMQAdeqcOVfEauTkYmAABM\nKqbMe0Rlwe4l0uHIMGZ6gnczy77Gxozqg+/vwc4SNrW9lt4EsqdxZ+NcQcC1sWr4i+xo/DoJtY8y\nSnz+Z0KIhoaz6ZBfnyoczCPE8d22hM89uYtQPegZlUw92BShFwUWeiKREbv3tQOAGzBLdU+gZAXh\n7bXXRyqLMdcGm4RCNzNx9l/DAQCVvUQ5SSkjzVeflWAhuh1ZjpCb5sE/gfQlPiR/br8BOc251YgK\nLnOEfUweLU0RY5DPOxH3DmUzIZaR4mensmmfI1iMYW2nRvRPWQCABc14naVQjK1O4H0D0oGKLdTv\necnUZZUo8zGUqODrwaZSsS9Tti9vZv3ftOUzAABpsxxIeYYLb8+Zlqi1Ne4Rn/BJUNlUCD/KiH3a\nwwGwh1F2Ac0FvD9LNGZ2SNCpqHOGAsrs2R4cv9p2FRv0le5OQMuPiPIquoX2WB7EDEzsMmYTcm4z\nYPVhNgHbFkV00oMtDgEAnJvC0f0TNuYrXsH3/Popm+TaOwpobIQHJfuZLbRHeeqHDw2cvG41Kgot\nN0Yam6/VIu956sDCSSyVtN3DspHY1DKsSFoFAOg6iFmy5u/xunzRTM8W40XoBfI2N4r7mdyKyAl7\nJjPoR84kIeAKs6PpVwh/Rm+uo5W/9EO4aIpZ3JX3/HUXkYYqAW0OHlmIhLm0t7XRapQ08p6mGjsQ\nck5CZQrpW+GtAAAgAElEQVT9CG+QCRXsd4hNo6gL5l/oq9giZRycwuxptw0c0Vqyg8guSyYzp5Vd\ngPjlYhSvOQEAkPio0FMzZVq2Mg7l3Sjnt0qJXkiMov7a4t0IOkX3zv0eUTBrNlMXNaIRbpvJF3F6\nPZEQ15opuggAjnAVUh/3h2QXvocLiNzDLLP2A4ECOUD/9UgnPY4XENWgEkhnTZQYapBOGUXvtUH7\nA+sEgn6gHM77iFDxiQStf74Pspp65pNFOVEF9V2nASo8vJe+krqXXUgbLwdSR+MDa1Gzjpl7eYSt\nPtjQoMkdKSP/JS9QSn5tXdUTRvGzVjS4j1gmGg1PsaJ9BMvBs6voR/p2Ea3uFfJJXmRHZSvuh8ZR\n3A81h3jeQD/6vkE6N+zbaQ8dHXi+2HyC2XGLRYImhAayYBKfrzlHfS8RMJTR0/Zi+VmeOV7TjRbf\n5LObY0QDJ0+wD8XjHNBdoy4kfpaJgrt4nihrT3kmmOnrexwmzMq4GwAQn0AbOPxfzwIAVINod112\nLfz3Ue72kdRFySRQ0V8SiVvUG1A7eG+NlvolH+f58fmp6/H41XsAAFmjxIjsCsrznD/92vHzt2LD\n5IEAgGcm3IeCqo/rhxkNlCQAKg/g7Ui9UcsSDIdFdUJToiJLtlO3VL0qb5wPJ/egT7n3RaJot4UQ\nEaaPkOC6k3vk+Q30I688TZkmfC9God8lIeE7+jnFXbmHvnKROuU9FgR1O66ZKjvt6skDPPhEH+d9\nh7y/H1/Mp2/ceTJR3Vl/8Pv+LkJFkqRYSZJ2S5J0SZKki5IkPSn+HixJ0m+SJKWL16A/+EyF/mJS\nZPjPIEWODZ8UGf4zSJFjwydFhv8MUuTY8EmR4T+DFDk2fFJk+L+nP4JQ8QB4RpblU5IkmQGclCTp\nNwD3A9gpy/J7kiS9AOAFAM///26kNbkR3bUA2amsBdZXqlCXwCjg3b058nbtBUZ1rxxJwCsaIlNM\nicxc194uGh6mMcVS08eHTXPYhO3zCmaBvt/dFwAQ2Z7ZtiuPaNF0BZ9xfnt7AEDE64xOnv68A2qb\nimY3KhHBrhNoCfGy+ExvRG0SWfi2akgNE6FSbzKEWoYm0AVnMGWQfQegqWW01k9MCBvZ9+SNt2+7\nyiz25zkcT+2fTl62HXMZAHDY2QLhD7MZ2PmrjFRqY5nR+bIr6+lUkg/vjmNXyzIxprB6MKORYWdV\nUI1jdsgtkgbBAbzeEsbM0qOxe/HiFdadH597vTJ15f/3a/5Nqd7kKGtUcIYaYShmRkvtAryipYVu\nNvtnLG3G/g1jdj2OzDRmsfNiiHjIGEfToStjTNYV6ULtt2KM4ymhI+Ll07FsaPv8gmkQrW8Qyz6z\nUM/gs/JKgyCJhsV+XzJy/UoGo8T+eVTGYaOPY83TzLJG7pFRbm3cuqhxAMEXJKQ9yIUf16IYtmPM\nVJYW8m/zR1CH5j07AcfUjMSHdmdGLmoEI/U7j3CspzzEg6Vz+P4385nxPlVAnVywgPXJY398GpEH\naU+9Oj5jyf3MrNnDpRvNv2pbMOMdsJMymvzMNgBAsTsAP15kb6/k+bTnOb/Ps78j1d++WCMh6jcV\nCoaQHyqPP5xZYux0F+qXyinqjl81Y8DJaQAA42na3Yy7+D9LClFkzoshyBku+ihoaBuDj1LxurxE\nBFEP/wwsXkP9qm5KvdUZacCL4/Uo9OPffCbK0S+YdqJ1BHsJnMiIh38rgajJ9ULVMNs21JsMpWA3\n9BOL8HwcGwYvunQHWvcgmq6iA+W09IHbAADeGXYM/JRdvAxMpOG+GdSPVR+w8b5K70DuUMrgzRFr\nAABvr+AY1vvv/g0A8GW3AYjbSP0KeY795vJqaZ+TEguhTqLsbR8ye+oewd8HdmGT/2KHGdYWXHMh\nx0QjzN9l2d+S6k2O6joJwSc0qBtC29iteybcd1IH838kmidoOHXAsSYC4UeIBMx9m/5Qwnze5+p4\n6mRBXz90GJUFAOgUQO6e1XMk+lcPE4Hw6MIZN5qd5mcSvaILpi4aKtU4WUUUTA2T81CpeW+fkUpX\nbTfAJBrUVh0KgWxtkH3+6s9HFRR+gHwI3Z6BS6/xLNG1XQYA4KKbe6FxrxbnWlE/x/SgbVzfkWeQ\nEa05CGFbaEdIZC8i/AUCKJvZ7YdTqIufpA+CJZO2MuIoXytaUbkr+jkQHsDr5JVEN3kNlGHi/ezf\ncLwiHpoC6nvukWZ/5Ov9Xan+/JtKFcLXGlA1kee1ug6xEG1p0L0fzw9ZNUQTwV+NvJNEmXjCxNST\nrtSPoUmpAIAD6zuioj/1qlMYfcwMf6JPOj7OfnCWuiAUreZaqfKjjE1ib3thyf2QBbovsBv32lui\nee8rVqKTjtY0Q0F/+kXxW5woq/7rhr7UI9XfOUMtwxXshdZLG6q5aIIsnAWjhntP7PAsAEDpsniE\nC6BkWhsi2E0XaWtzniMaJSBVDdtK8jdYzTO+bKOe3zaP/TUXrR2GUvbZR9xWxg4K6xj76TH+HJzC\n2J49RTusEq0YW73Jniy5jiCE3UNbXTgv8XeZ9V/pdxEqsiwXyrJ8SvxcC+AygGgAowEsFW9bCuDP\ndW9R6C8jRYb/DFLk2PBJkeE/gxQ5NnxSZPjPIEWODZ8UGf4zSJFjwydFhv97+lOhbEmSEgB0BHAU\nQIQsy4XiX0UAIn7verdVi8JD0Uhez8hS6iP+MBbwI5yfwAy45nVGr6aM2IUVP7JXSpCYPqDfwOzL\nd6/PBQDctu1J9Fz7DABg2uDdAICUT/iRPi6lrP18QNZ9YsyIxFDUT0sHAAC8UYC7hRgHWstsW60Y\nWZawmVHO/L56BD/BLvt3hlzBJ6tqfu9r/q3pZmWorlUhYI/xRqTWU66FT0PeGUsYxu/kz47Zb58a\nAXUG6+WuekQ3cyYDcH8ER8qd2J+C7EpGDzsnZQEA0jazB8vM4xxr5QoAXJN47ykD2Axl6V4ikVQe\nH1ZnCdSJCAbHmrm+LmxlFmJW+H0YPOIsACCnn0Cp/fB73/TvTTcrR5XLC2NeLfKGsR7b2sqJWd0Y\n4f32C6IT7kpkd/QBPS+izsNsy5k80fRIgEPuuo0Tt6L1lWihYzT5ZTcz35ZfOZFn+pb7AQDBNhkQ\nY+dyR4npWpWMNscs1aBmBnXr3AHaAlMr1ie7Qvmww2VNoW9N2frtMELlbpDR/xt0szL0agFrrITE\nH5h1KX9Jh07TzwEAsp6lDr13gMiuh97bgI0lDNuPCGMkfmk2+xKZs8T0ij52jFrCzLkzWkwr+ZD8\n/vp7Tn2J2eVGYS+uhfvvYnbu67OiN4NWRtlOZhKmT+I0rVUnOKFt2VwxzSkZ8IpJKHWxzADh+O99\n07833awcJY8MfaUHEftoR13jK+CrJG/C1lFPPAYazvBfVSjtQBtmi+X/dFWUn3c7s9uqJjJGdqW9\nO/MeZV7Wjjq0+TLRSFuzukLVX/QTOMZ98a5YIgsX/TYSksgU2Xtwf7y3Bf93YFpXAIA8TQN7uJhU\n41DfyLA3VLppXazRomp7JD4ys+fNLU8dx7GPuS85AyifwhnMUpsP+MFBUWH6+F8AAAt/oM1t9RBr\ntjULEhE9k1OBit3MomoEAnRVJvvZJCSUoDaC+mb7jpm0d15eDABYU94NaXPYxEX9NLOxcga/xvHV\n7FGmtcowBXENBGS7fu8rNgi6WTn6tIA9TIL+IJGwaatbwzuN2Wi9mArh9HCxd3v0NLZ2JS/VdtGj\nKIK2cewAjuBcu6cH0ipEbw0vrzOU8T6zXxSTQm63wyCmcgWfpp/k03DvtEYBheeZMU/pQb+qaBV/\nt0VSdtW1GlQNEnuhygOfrnHviz6vCnU1BsScY3b6ytPNEL2TtvKUhWgfXSf6GvZLZpii2N8hs46+\nkPk0kSI1yXzVN61Fj2jyfn9mc16fSDm9vZ59O/wKJAx+cw8AYMmx3gCAf/UlwnfeorFw9qPuR07l\nfa4W0wBkf8592j/XCf/ZRDtFfC/6cvzeF/2b083KEQB8Gglxr3IzuvyEBMnAs9z51UQXtB9PFNGV\naSpojlLPPAGUTdA5vha2ov3sf+cpZNSQ7+eEHxuTxvsdW0pUkn+hF1EzswAANQX8iNFt2JOluNYM\n7z4iYoxacZa8wv1Vc4b6quleCWeQ6MFi90KSG7cuShoZhlA7Yj6l7eu+YD/GWjhB984tnMbUqyNR\nWmntZJgzuVeeyad8zIPJV5Xoa6N2yXg0nmfArx/nXuvfgffeuoI+qvdxK4J/oj11v0/dtpXQtzrw\nW1uondQvexJt9s9DFgAA3sgjgvT4laY3Pn/LC2W/9xX/g/6wGyRJkj+AnwA8JctyjST9G24vy7Is\nSdL/uHIkSXoYwMMAYIk04r4xu7Dl0gAAgLZSglFAp7qPJQy1NJewyt+KUxB4lUYwJ5mLWCt6592x\nmo3cND4goDXLOtbPZ0mJfbxgVjwXvP8VLUyXaBidnUTD2nQqaGEvNbxOKl3gBbLCFsmvkTeVzI4N\nLcH5SzTCa0Ztxg9qxx9j2N+Q6kOGmsAgWOMBSwfKreZSKH4Yy3KAezazeeibO3igbtEyH5nZhOHp\nDZSHPYoL+4VLVAbE2+A6I5rKLuFp2/YU5a6t4efz6oG4bQy0bT/AQEqLbG5QQfPzkV5BI2nJ4HUn\nrolSsS48DM5ocQTffs9GqcGpDb/5Xn3IUR0ciMtPWBD3M/lhD9dhWSZLN5wi5uQL4P80kg+nDnPj\nj91BOcrPsGzk4Ks8lOsrnGi7gAd1q536Vi0aPSd/QzlkvqxBzELK/+4n9wMAlq6+BQBQkQIkB1GX\nX7qHo+nG7HocANB1JA3uqew4+ETJnb6pHt5zDbLkB0D9ydDRxIPSF+gUdAgrxJmv6ODb+vJ+thb8\n39yLQ9DkS8pj3hBuGImLGQCroU8I8zozxr3I0oOf3mQgpHSugGrauCjy+2qR+C3HdlrGUQeT36Fd\ntX1SCZub8l22hKUL8kg6tc1CKFtvtQXOw3RcfY/w+Vj3Bxj2N6X6kKPOLxAuiwY1Tclre4EFx0ey\ndmB4yAMAAGuR2J/a+6AR5UCRB8UBL4Cvtiai4aFVgklDuftnUTYV9/J/lp08KLafcgGZb7CJeMUj\nfM93nzHopXfKkO6ifa+rZmBnRRoDKe7x/H1st6NYv5v2whoPeHV/lGN/P6oXGRoDYbnmhekJlt4c\nKGgG2yjuZ00C6dT5tjD4UdXRjeQvqTvfVDGQEnWOfsWpJjywqTtL6GzgoW/r43QU7bdzf5MPUX9K\nulYjpJz7YqlwKl9670EAwGvPL8WRJJY4h4rmwf5t+J6Q4dTfbqHZeDWMB/+BZ1gSi1//CMf+nlQv\ncgwPgH+vUlSfok9h6+5EWxPlkC+Tj6UlTAJsy+gAScNbPt1hJwBgy2wGLFefYTAt5IKETr3ZTDj9\nFQa4avuIz6QWn69Mj7itorH4NOpn3BbKuqKl9sYI5isn6Nf4Won6E5HUMqer8fgjGwAA75+4FVA3\n3ENcveyLoRZIahmtvmE5Rl5OCkJ7cP8pPMm9L7Idf5cPGOA7SpvW5HWW6OVY+YiDGdTF8C16nJ/I\nUtrEt0T37c95yE7L55myuokKmz6nnibcXQAA+OD7sQAAVyc7gtTiLPNrAgDA25r3qUqkzS/rYIDv\nGvdny3xxZh3y+/z6u1J9yFFrDoI1RgWPUQQxQmthMZFvd0xjuOlwBcujKk+FIfowz2zxb9JfPFjF\ns+SlEwkAgFS3hMkjmHh3zqM8rTG0iVUdrwdEtajcyjXSbRTLiqrup+9jfTAIchMxaKGYf5PtVML2\nwkfNt1pgE2XtddEG+C423EG69SFDvd6CmM80uMZJxcjY3wtrK3iGkyzUiby3eLAPmVGOcpl7W6zY\nM3M6iURSM+qrIzscL+6lXoXGUXb2SN6npinrwbwex40SvbKfWYLui+MfmpzywSdGbkfvpV0dpWdg\nu2Us/dHE+GJcO8eAzpVHWKKHWb/PL+APjk2WJEkLMnaFLMvX3d9iSZIixf8jAZT8T9fKsvy1LMtd\nZFnuYgpqwF5XA6f6kqHaZPprPrBC/yPVmxz9FTn+X5Eiw38G1ZcctXr/v+YDK/TfSJHhP4PqS46a\nAL+/5gMr9N+o3vZFs7Iv/l9SvcnRT5Hj/xXVlwx12sYlw99FqEgMS30D4LIsy/P+y782AZgC4D3x\nuvH37lVZYsb6zwbC0VxkmasA3Q+MPq4cw4i+/hAj9EkTTsMzi1Ep/5mMGF4bywju7ns+5DU17VHt\n4Qb4Q2eOV5rUl6Uk3+9kFAy9q+CwM5ATEchMXMlkBtaaBlci6zARFH7FjFa5TSLsLyBcef4mBJTx\n8x5wGGD1NbyseH3KUPIyA+r5WaBCvMCs2USmDHueMPM9GwlJLrkUh8S7swAAFYuJ8rEPJurBdZCR\nyL2Pf4AFKZTdYTGONWov5VM9RWS3gyrg60G+17xPeWWITGkXAHNaciTz47dz5FbSfMoyczShfgsq\nBsFPhA5zh4ug6vrf+6Z/P6pfOUrQVqlRJ0B7fgUSHKWUaahAcBX5M9R+ckk7xN/FrKa5GzPfV35j\n9ubd+csAAK+cH40nLZcAAJdfSgAApE2njOwfMqvQM6Ac+x8kNH3xecJiAwRCbVB0Ooqd1P1xRx8C\nADRdSVmlJ7F0y88AOMTY1sqWgKcBZlPrVYYaGfoQO9yHaUOvZFig1ouGhSIKrymm7XNWa5E7lVlw\n/Vn+7/JsyrtnG2ZismuDEKOjzfXo+Z6qi9RTXXuuicHDTuNgW9rjrxeMAgCoCVJCZZkdP/RcBAB4\n/X6iHbIFrDC3mrpYFyvj0YlEwRyrTgAAHPm9L/o3pPqUo1cPVCap4Vd4HdaqwaD5LL2ytqfMLGcp\nx+r2LrhCaN+cZu5VZX2IGjt6yycAgO5bn7rReC1nOPluMlDPXEKu+zMSoetM/ZYFQlAj4MkVgxzo\nGsh1UJoTJD4jMzxqMdb1wEfdoUmmUXWGexpkVrw+Zah2eGC+UgXHe8x8Vt4DqPWUU3YGkQ2RheSh\nNV6NzDFEOWhECWTWw/zf5l4cs3nb1ifRwkh/9cAzRLOkBDCDVm6nz+NwaaGdwb850/hcr4EyeX75\n/ZC7MMuXLZpjWjjxHqU7mX07ej4cg8J6AgCOv7uQ3+P3vujfkOpTjh6PGmWlATAIWHjYT3pU2BIA\nAEYVbaAkstL6chUMnVmm8f3bRBrZ7laJz8R9sv/0o3gybB9/HkeEysQu9FFLXfQx9/zWAfmvU38i\n9NTT7HEsbw85IKN3ylUAQO67RIkW9OXzw0/ymvLWwIebORZUHVcHqeG5qPW7L0qAWu3Dnq+IoNOq\ngbNdqDMWUd5alUYdsPWU4AyjnhZlcq9ShwvZb+V5o8+zR2+UA3kWipIsUfMcEsQzhdWuh3MYbbXx\nbqKNWm2mcp872AIu038i25t9zVeXhWuqpIsGQUKeBZGW3/uKf1uqTzlCAnwaoLQXeaS7bEaFRB/x\ny0KigQJP0bYtmvUFnigj0iDzHJGXwQQcoUKMlvfZ1Pg+lUhLfSL3vusVCeoqMWI+QMbkO3YB+HfV\ng2sEZW3KA6Qh1Hev2AdtGq6R/9femYdHXV19/HtnJpNMMpnsCdlIIIRN9l2ggCwKomKlUq3bK4oU\nqxWwi11sRd9aa92qtoh9tWpLW8EFwRVQBEVAIptJICzZCNnJnkySWX7vH9+b2L5P3+eBQgd/M+fz\nPDyThF9mbu73d+69v3POPbfsedpmWLsfSTprLXLpKVj2m28r5fnU0OO0oGqSA/Hb2Cf143yw6goc\n3XrNUHkz+8hT60JULe2zehezOaO56x/RQ/hLjUkGVk3jw9szadRnaLQuw8GzDlBb2Be136K9pb/M\n+8OdTH0r5vlgr+HXfXUiWFQ0r219nM+WJ+cqRFWyHY8s5YEaV51hhsqZbPmZAuAmAF8qpQ7on/0U\n7NR1SqnbAJQBWHRmHylcAETD4EB0ND+iYXAgOpof0TA4EB3Nj2gYHIiO5kc0/DdRRgCL5sREphmT\nBi9BzYP0CCc9HI7uh5iFMLcPo9vP7aHnMSrejYlpLODU7OHeqH37GRV3VNNLbFiBjI8YvWnL4DUd\nKfQsNY/UnkGfQp9tvL5xiM6MGUEPcsvpKKgO/l/u3Ty2+fiTDLfa9JGsUSMb4M7TUdoxjTi+8n/g\nPl5pwhjA+SEiPdPIWroSVy34DADw/iuTYZvJqMr20cxWuLn4SgCA3erDga30Fn+6mFlFC5etAMD6\nNQDgifcheSe/TtAFu+Yls57OgVZ6DAsbU9Cyg6kUcdMZkav/nN87y4CoWt5P5fPYRlsr3y+atYQR\neXUNqk/T63/JABb9e3HCy18YhjHunDvEpDgTMo1hly2HJ5K3smEFnNd+dYQxAHwwlcWabij4LzS1\nscjTtkmMZH77brpsx/xiHwBg84YJSCigDrWj9VGvgxm98ZYwEueN9cLqZDR9RAYzXsrWMmNF+YH2\nS3m900FvdMdnzKCILtfHYkeq3qMjcyeWYdcdf0dzUU3I2mJ4ZqaRvnI5rB26PsYJIPt23t9Vj7Nf\nK+ay72yNNvii+PWwEbQz7zJGyQ+v5OvAO/bDu5mRu/Blen/qH6ipzcLf9f06GdYufn3iNuqcsJ1R\ngM4kBZ/eP+zuq8+i0xl99nrapGVwG9QBRpk6U3UE/3s/DGlbDM/MNNJXLMcl32ANor1/Gdn7f8Ys\nhmjaWjm/xcR0IEYfb9zUQZsMX89otjuJerT298F1lP3tKmd0rvlWZiukruI1RXdEwVlCjaMqqWfN\nVL46j9uQOb8UAHC4WB9XeIga23RxuMT9baieTB1ts+pRtPxFdByrCl1bzMow+vzsHlw5bj8AIO+x\nsXCdYIS68mfU4LK+rOnwev5oZKUyA+jUXvavXdcLm30t00gKlg9HyQJGP8Na9XiqlzRRldSgPV2h\noy/fO5NJXzg9hJqq8c3wHeKc1xVHXSMzeQ9EvUl7NyxA/QS92VwfDVp2d2jbYs8atWw+bSqsHTBm\naBssY38m6Ay/8EU18L/E7KP6BczCVCeYCdGti3rbK+1I386xsCOJ2tQzgReTLub9sHfbEIQ36KyI\neazB0/YKx2FnRTfKL6XtxbJMA5p1LUHLAM6XRpET/d7S2UjzXShb8wQ6K0+Gri1mZBoZ96yAN462\nEVUcBp1QgD6z2L8tnRxPW/cl9PZ9xwQ+S2S8wkms7DraRszn4b1FpO+5jkH5R/NYIyzlPWpTPdsL\nR6kuaaBNyq+zRbvj/Ejox3uosYjZpD4nDS7zvZ66Vz7YT3Ncr13Fdh+88lchbYuO1Eyj/y0rkTCb\nNWnCf+HC6eHMeu4ZC+vH6zkrowVWXdLDr8Vu1fbaU+nD1a8Jlne5th1yC7NyD780BADQOIzvE11s\nhWeqLlhcT1u26ucJn9MHawfH4qHjSgEAZW9wQRpTRs08DgvqxvDzY48ARzY8iY660LVFR2qmkb14\nJRIOs398dgvqRutn655u0bsHDAvQkUaxPCkU2HWAc+Cae/gscv3mZUjP5vNm07Y+AID2Qbw2qoj2\n54386ihkdw6fJeyV/D/lA3wDOFbHv8f1U/1ofd9E8B4Y9EI7ipbwPhs/jMesvz5lzRnZonkr5giC\nIAiCIAiCIAiCIFwgAnrYoTfSivrRMXDvo2fq+Lf9iF1PT9Dm47omxzC95z/Ojt179YkVA7S36gQ9\nhW69781nN1A1Re8nHkXvcuQefm9p5p/mj/Ei525GAnaXcO+/ZS+9lH9a/Bxu330LAGB2Pj38DaXc\nI9exk+1pKo9F2mRmRXT7rLD868LGIUNYq4HUT7uQ8i16cWOPeZF5Lfd6X34Xj8HqOXEg6YAXXfPp\nKhy/hUfwWmbShxfeyHvAAyAhjx7HkynZAIDnFV+jT9Jj6E62wKadmXV5zEzJWcsq6zds3IY1P14I\nAJg4knuND1XpqOoOeiAnpxzH32vpXCxuTTjnPggGvBFAwxALrMOYIeY/GIOGIvZtzBHa2fwTrOPQ\nHWNg0jiGx2av/hEAwD1TV+j+ktH0qZfno24jo2opK6jN/lJ9wpOOvn5v/ha89ktGdo4MZpjNewmj\nbPYDTiS/Qr0yf8JTEQ5NYDvau2mv3bFGb9SnaU1f+OpDu8i1xQM4qizAFGbcRW9z9I5PbWm6GoKP\n3xthBsKadfbeb1nPqOYBeuqT3mVErv6tHHS/z3Hvh5vWAQCefYRHALX21QY4GUgo9H3VAHwVMY26\nqAHhb2ut4nqiOrxP+q9nhK54URzSZjA7qUofmR3q2FsMZHzow84m2pJ7mAcRp6iJdRf7099Pn04W\nA5QdZWTGUcE+7mQyEsJGsY8jvFbEb2IE9pnnGdm57tl7AQAtg6jHggl52J7PmlURDdTTmaJPWet0\nwf0b2vIvn2JEdpXBejlhdWyX3xrde0y91WIgZENwGlu7QtIuKz5M4l56/8J2NOusLncT1ySvl7O/\n75q5Bc/s4f7vBL3Pv4MlULDldV5jHwkobburb34OAHD7Bh6VkPYpo27zV+7B6yW8ZxoG8T5ZcuO7\nAICDrZkofJOnXNTpuJrnMO0tolFnCP+4GD/vw0zTFytZc67sXDvC5HS7rKi8JBbdQzk2OlztMHTE\nO1GvW926xkbC/VEouk2Pb3/UNYt4OBqUrg/gizDQ0pc2M3gxo+K2x1kTrHQXXyMyFKwzmbE0K5nz\n7Mu5tL/GwXYkHuJ7tSzkGtXTQtvu/xznv6o7W1Fxkc6G6HDD0JHWUMUS4UPEwGb0eYLRbY8TCOvg\n+NnQwH7t0TC2xI/TNBOkrw3T13OejDjG71v7Gcj9C+fY38Yx+9qI5dwXWcNXe5UdEXVap28w0yR6\nF9czvgFdSLqVY3PLPXr9aeiMzW7aYnOOHS2L+fXU5HIAwMFz7glzY/EAkVUGGt/het42woAvgrop\nnQkbv59anQ5zYuhvuO7068LStRN0JsRCXXtqeyo6JvJZclcxnwUjEnlNdAn1uOSmz/FWHo9Qjkjg\nGJPhFEcAABLUSURBVDByONcrhW8MRnQ5NTqcyjnYNZvPLjUHqWvyPj8yRzHTuzwsFT4T1vk7n9ib\nvMh6oxZ1U3haTnx+Gypn0C7C9HNGRDP7dPz9eTjURPusfo/PDtZu2tT3Cq4HADiLbWg5pjNTBtL2\nep713X047v1y3mtYV81aOQVHecpPz46TsDbAUqXtUj8+aFNEVDnf59TPDcTbaO9HXx10Vn+vZKgI\ngiAIgiAIgiAIgiCcJQHNUPHZgfa0r6pqZ2w24I6ne8gbydfmi+h1iioJQ2cSPU5/u2QNAODhAayk\nfuqv9C629FfoGEEvoutTeiXdM+jFj/yc+7u70rqx90Puk5swk5kqJe/Q63Tvo0uhdE2GrX14jU/v\n+XfU0zMWN70OtfsZufcke+D1mLEO/vmj26Vwco4dz3w2CwBgv9iKqnJGvCf9mP1rXcX+PTXNhjA6\ncGHt1LUURtNT33WQEbWkXVbUTGNUPLyJfT5hKfeh5zcwbJe8ygXbr+h9PlHLa088SL0fOng5/JOo\nScV+hsotbl0pWnuoXysahcQE3hd1baF1jNf/R3RMB6bP24+dr9MbH+4GFi3k6QNbc6lfawMjmrlP\n+fBFK2vh+BJpk/kLnwYADF/HrKQ9Wy9CvI6SlXxEbS25zBrLfpURgk0zRiD5+wzJ3prEehFPvXAN\nAKAj3Y+TQ3XF+81DAQBDZh4DALTPo3ZHj6XBkcj3bGhxwbv9/PSFWXHEdWL4Nw9j10He976Lbaj4\nmNp5BnOMXXQxazK8vW5yb8ZXx3fpfZ+SyL3Je67KAgAorw3tWfy9/97AzBRQdhg688WX2oUFi9jx\nz22ZAwDw62hRu9sOzGKWw8AHGaUr+gnt7cjdHI/trna0dDJyOCad+9mPnWtHmBxrajdc951E83pO\nRhZvGDoyqEPOEGo0NZF7eTf+YToGb2NGYMUVnJfcg5ixYNORdO/RaHQs53h51RusWWVN7Dn9iWPj\nB8VDAAaN0DyUPxv4MMfRqikWlM3n9Q/vZ2EqayRt05uu63GMbkJNPutHJEd0wmoJ7ai4NaEbcTef\nRHM9ayRY851I/YQ20JLNUNhF380HADyXPxU5f2Z/nV7B+dDXyWs8bhqTG0CEk7r+upQnZs2dxnnx\n4zqeiPjqazOw/DsbAACNObSzNa/y2oRCHxx3MFJqb6LtqQK+1o3msq9qZy7uScrmH2DC0wv/Iyie\nutX/WX2qVbsFJQupad/jnHvqx3H8qpgZjYEvcm46cS3rhI2bzAyTg5s5cEZWGejUUfBjq7nGrJ1F\n2w5ror1ZuwGH/vgOP++DsPavIqo1l/E+cG1nTYiBC5jBUHwpo7iWw3a4+9A+bQ02oDu046RRYd2Y\nkFqObUuYLTYquxTlLzONL2wBx8XwV7iOrJruR+5fmLVw4lvM/InQJ414XHrMdPhxZFm0fnfeF5lp\nzGS3dFOTsFaFOUuZ7XVwCVNeyu/js4yvKgodE7IBADZd72zZN98DAPzeN0+/D+Btpe1/XHB2UfFg\nxevyo2GeG5YTtI6kg/7erMiWm5gh365PcO2zyYGGicxcSFjKPLulKR8DAB75fC4AIHFKHaL1Osb7\nJie/FT94FQDw8+1ch24tGwRriz4FZgDH5qK/0pYTi7pwejhtf3wWP+Pz0mwAgFOX8rvqwa34wy5m\nH7qym2Gx+85DT5gXj8uGysuS4ZnOTPj6CVGw6Do08YXMTj96E5/l/L8ej7ZUjold6TqzejJ17uvk\ntcdGRSLWxXHYu4s2PO4Kzqt736fd5bX1681MGTuEhTRPHOJY4HUAzTnUJO4APytthz6BaKT+zMIY\nROp6VS1z+Vl4+sz+3oA6VGABvNEGDH2cYO3YMFjduoDQPDpGwvWljvRmXJ3BFMkHFt4MAPjuOp1+\nbOXCM3JQI9pPcED7zp3MrVr9BYva+pLZOeH7nejI5Of1j+LTfWM+F6I1U+KQ8REHvWLFyUn1rAtn\nsiPbd/eB66Q+xjfLC1hCe8sPrAa8Lh9SMjjYuA8nQ1VzMXHyT7xpL3+cx46t3jYbVu3c6LuKk03f\nPVz47dFvVx/nQp/PeA/UzOfC4dhKLjxaR3Ag9f+iGnWf6Yc+PT45qnUxr04DTXr+iT6uj7TTqVzX\n3MAHvx21A2C38BdHpDJ9r+DcesH0tHZFYEfZAPidehCxKLyyjwWZB976BS9axO8rZlnQncAF26Dl\nTEQd6eUWLr+L/drtAFqyOJx0ZXCBEtazPe55Plh0dUVgfzl1zHHSFmNK9HHlLgtyfqftcyJTPL90\ncBEUqWtA537ejuPf0UXJcjtghIf2Q5xN+ZEY3gaEsZ8ddQb8dvZV6pPc/rbpOU4y3pFtaBmrF4MP\ncpTdcc1wAEDyPv5+7ViFeO3d6Irl+3TH9Ozr4EvGGzY8X8m89iGTSgEAVX/LBgBEjWhF1QEuahrG\n6K2Xuk5pIuc8xB3uRPlcToT57Ynn2gVBgdttx5eH+yLjFG3BuqwGnS+zH90f0xY2JHN+8kUBvtUc\nJzNv16nIw7g4TPsN7e+qF9/Gu/NGAQCiXqBDpqSQzumIOl2EPbkWznRuratoYwHOkqt5Tb+Ly5D0\nEOfIhkHUce7SnQCAja9OBQB4t6TAnk1tT6/PgLcxtLffdXXYcWJfZu9Rti2LmhH2JvVsnc4Htf1r\naW/ewT4k/4rFo5v+pB+8dSp7a5Z2XMZ7oPbRoV0axYe5Mi8DF5k7uVYqu9OPxzbwuNyedUvWh1y3\ntPSLQOd66hnr5nvOvJcO80grx+e3H52B6FtrAADFh9LPSz+YHX+4gfYcD06k6O2nm+2w81kA3bF8\n4I08RRvqjjNQvLwnwEZNDq+jnhF6h17LAMB1nP1fN4d22+d9Xch0Gu+PtLcMlMcxCLGugPbV7xGu\nl7rmj0fKbuqlijg4187RgSY9bteNUVDaieIc3AirI7Qf4lrbHNj+6TCkDaPj+Wh9MnwZer34EZ3A\nbl3M1H7aiuqJHONyf8jgQ8kD3C5gZFJT1EVgyCoGgkpX63FxGzXIbOU6eNVtb+C+PD6Up2ZQ34it\nukTBSB+a+uutknqYLHZzzDb0EbvetC7YSzlORI2ks6b8nHvC5HgsQIUD8YV6m/cAK/za3LqP87kv\nSS9VfTeeRkwE15nlGxlw/90UOkKhHVW+eIW23ez3J378AgDg++sWAwAcA+kYte6IgX8A7aeiiZ8R\nX8nvayaEI+Nh2uXRNh43b0/RgeIWtnHj/bPhGKqPVf80Fpam0A7A+xwGmi/ywNpFDRLyrL3bthou\n4nPjkCe4jmkfmgLnfDo82wpop8ZealCUzDkwos4CRyGNqHER57oeR0q4PmK58AfDsejJvQCA1wsZ\nMLZkUJ9x3ziCmvvpP/Dp8gEfv/BHAMDAl5YBAAY8UoDqG3jE/cwB9KwUn+HfG9qubEEQBEEQBEEQ\nBEEQhH+DgGaoOKPdmDIjH3s2MVIz75rdGBTJ7QDr72Cxyr6/pRd+jKsMz+TPAABYrqB36idrmanS\nOZzuf0txDHJHM8r24jr+vtJHBHrjdbpduA3w0SP29y30/jun6+JifQw0TaSXasiDTI9tHkMPdNdR\neq3ti6pRmc6CQ3E7HLC2hbgPSgEI82Nkoj729ko3Kt/O0l/TG3tHLMPR24cPRJePt1jN9ycDADof\nonbuqbzWZgDVlzNy44zWx8axfiI6C+ixb66NBWKoa2wB+z9uIbcLFB/tg3B9JGvqJwwlNQ3m/ZJ3\nOdtVen9K73GWW166WP8h68+xI8yNpcWCqA+cmL6MuUL7GjJxKo/R8OhPmDlQuYl9/fTta3DHpiUA\ngFN3M93c4qXdXDqWW3fqOp0YM4W22Oih7Ww5ydShkm3ZAABPtAG4qP+VsTp93cUsmOR9PhT+iFH5\npCwW6Mt6jDq2ZDGjoviaSCy7ZDMA2rvqDG1bbPfYsbsmu/dcwOZBBqJLdLG9t/RxyW8wyh0/pxqn\nTjFiUz6XHv7U0RzzcmYwMyhqZX8cu5l9bXUxKjo3l1mC7+xjgXB3nA05Exg7KzjMrAmM1wX6nkuG\nbwbb4k6kNv3eYhTh1DTeEy3Z0Vi8iNmEb98363x0g/mxGrA4PUi9l/16oCIdESnsv/ZM6jh+ArdT\n7snPgfcjZirYWR8RzkK+ll9GXT9pzMXJb1GbFEWbHDWKMZbqnTkAgLrf90NTGyNv1ZM4Rqfuom2W\n+PsiTqfcJh6ifm+t59zZe8ShFfA6eE1knR8W77l3g5mJd7XhujmfYq0ez2anl2L35fr4a72bpudI\nyKcu/TMeeJzF8Nt0QWELp0AYqZwDB964Hym7aLvlD3EcLWMyCkqW8X2sxZEwcrjFLmED7ct2hPdQ\nw12ZuH5oHgAg7zKmP3+opgAArvnRVgDA8vtfxWu1HM+tqR3n3AfBQFgzkPmOwsmr9PbTsXakfUpN\numNoJz1Htnoj/VA1XKP49bzWOpA2FVHFNYmjWsEbqd+8iVFa121cu9Qc0QUYL7bCcHIMjcvjNcee\nnggAUD4FZ5k+5noa183p1+4DAJSv4Dy95IoPcEvMIQDAxE0r4AvxLT8q3Ad7dhvGJdEWNhaMgG0w\n7WTJ8E8AAGs28HnhomnHUdLI9X3HCZ5nrQZye8GiXGbjvlk0FSefZ2ZDwivMkK2YQ72LVlDcB1ff\niCR9/Lzy6uOWS3mNqxxo0+Np51C+96FfMIMwicmBePnXz+LK0h8AAPxb5eAEAIiM6sToyUfxhY1b\nmjO3eFCzmLaYFce1ftcunVUZW4/Df2Nmu5fJXsh4nPb66Fru17jpwK2920OWr70NABAzlmufrBhm\nBe0bEAXXUV1+wkKtI2P1sfXjm4APOZZ6NurC1LpAf/31ekeDxY/OCq5b3Vk+eLeF+I4Gv4LFbcXA\ngfoggvBsOCv/OYOubjrHwfqZXRh8ry4Efanu8xnU5e5cbbfPLsCdv+Gz208+4rb0b8zjM8iOY5xM\nPdO7ezNT7IXMNOwpPPvFR4MRp+fhvt+lr2HanSz2nupju4Zsa8N42w4AwJ/3TzqrPze0R15BEARB\nEARBEARBEIR/A2UYgfOgKaXqALQDqA/Yh547ifjn9mYZhpF0oRpzoQkSDQHRMRh0FA3NryEgOgaD\njqKh+TUERMdg0FE0NL+GgOgYDDqKhubXEDhDHQPqUAEApVSeYRjjAvqh54DZ2hsIzNYnZmtvoDBb\nv5itvYHAbH1itvYGCrP1i9naGwjM1idma2+gMFu/mK29gcBsfWK29gYKs/WL2dobCMzWJ+fSXtny\nIwiCIAiCIAiCIAiCcJaIQ0UQBEEQBEEQBEEQBOEsuRAOlecvwGeeC2ZrbyAwW5+Yrb2Bwmz9Yrb2\nBgKz9YnZ2hsozNYvZmtvIDBbn5itvYHCbP1itvYGArP1idnaGyjM1i9ma28gMFuf/NvtDXgNFUEQ\nBEEQBEEQBEEQBLMjW34EQRAEQRAEQRAEQRDOkoA5VJRSc5VSRUqp40qp+wL1uWeKUipTKbVNKVWo\nlCpQSt2jf/6AUuqUUuqA/nf5hW7rhUR0ND+iYXAgOpof0TA4EB3Nj2gYHIiO5kc0DA5CTceAbPlR\nSlkBHAUwB0AFgL0ArjcMo/A//uFniFIqFUCqYRj7lFLRAL4AcDWARQDaDMN47II28GuA6Gh+RMPg\nQHQ0P6JhcCA6mh/RMDgQHc2PaBgchKKOgcpQmQDguGEYxYZhdAP4O4AFAfrsM8IwjCrDMPbpr1sB\nHAaQfmFb9bVDdDQ/omFwIDqaH9EwOBAdzY9oGByIjuZHNAwOQk7HQDlU0gGc/IfvK/A1vvmUUtkA\nRgPYo390l1LqkFLqRaVU3AVr2IVHdDQ/omFwIDqaH9EwOBAdzY9oGByIjuZHNAwOQk5HKUr7f1BK\nOQG8DmC5YRgtAFYDyAEwCkAVgMcvYPOEM0R0ND+iYXAgOpof0TA4EB3Nj2gYHIiO5kc0DA7Ol46B\ncqicApD5D99n6J99rVBKhYGdutYwjDcAwDCMGsMwfIZh+AH8EUxjClVER/MjGgYHoqP5EQ2DA9HR\n/IiGwYHoaH5Ew+Ag5HQMlENlL4BcpVQ/pZQdwHUANgbos88IpZQC8AKAw4ZhPPEPP0/9h8u+CSA/\n0G37GiE6mh/RMDgQHc2PaBgciI7mRzQMDkRH8yMaBgchp6Pt/DbvX2MYhlcpdReADwBYAbxoGEZB\nID77LJgC4CYAXyqlDuif/RTA9UqpUQAMAKUAll6Y5l14REfzIxoGB6Kj+RENgwPR0fyIhsGB6Gh+\nRMPgIBR1DMixyYIgCIIgCIIgCIIgCMGEFKUVBEEQBEEQBEEQBEE4S8ShIgiCIAiCIAiCIAiCcJaI\nQ0UQBEEQBEEQBEEQBOEsEYeKIAiCIAiCIAiCIAjCWSIOFUEQBEEQBEEQBEEQhLNEHCqCIAiCIAiC\nIAiCIAhniThUBEEQBEEQBEEQBEEQzhJxqAiCIAiCIAiCIAiCIJwl/wvj3w4qUrdnnwAAAABJRU5E\nrkJggg==\n",
            "text/plain": [
              "\u003cFigure size 1440x144 with 20 Axes\u003e"
            ]
          },
          "metadata": {
            "tags": []
          },
          "output_type": "display_data"
        }
      ],
      "source": [
        "images, labels = next(iter(mnist_test))\n",
        "noise_dim = 128\n",
        "\n",
        "def get_noise_batch(batch_size):\n",
        "  noise_shape = [batch_size, noise_dim]\n",
        "  return tf.random.normal(noise_shape, dtype=images.dtype)\n",
        "\n",
        "def get_label_batch(batch_size):\n",
        "  label_shape = [batch_size]\n",
        "  return tf.random.uniform(label_shape, maxval=10, dtype=labels.dtype)\n",
        "\n",
        "logits = gan.discriminate(images)\n",
        "noise = get_noise_batch(images.shape[0])\n",
        "gen_images = gan.generate(noise)\n",
        "\n",
        "num_images = 10\n",
        "plt.rcParams['figure.figsize'] = (2*num_images, 2)\n",
        "for i in range(num_images):\n",
        "  plt.subplot(2, num_images, i+1)\n",
        "  plt.imshow(images[i])\n",
        "  plt.subplot(2, num_images, num_images+i+1)\n",
        "  plt.imshow(gen_images[i])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "9yV4JM345J71"
      },
      "source": [
        "Print the names, shapes, and other information about the variables in our little GAN model to get a rough idea of its structure."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 12,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 615
        },
        "colab_type": "code",
        "id": "9ti32zve4_QG",
        "outputId": "4b3e61aa-a999-4c92-99a8-c83caef12ad7"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Variable                                                                         Shape      Type     Trainable  Device\n",
            "little_gan/discriminator/block_0/spectrally_normed_linear/b                      1024       float32  True       /job:localhost/replica:0/task:0/device:GPU:0\n",
            "little_gan/discriminator/block_0/spectrally_normed_linear/spectral_normalizer/u  1x1024     float32  False      /job:localhost/replica:0/task:0/device:GPU:0\n",
            "little_gan/discriminator/block_0/spectrally_normed_linear/w                      784x1024   float32  True       /job:localhost/replica:0/task:0/device:GPU:0\n",
            "little_gan/discriminator/block_1/spectrally_normed_linear/b                      1024       float32  True       /job:localhost/replica:0/task:0/device:GPU:0\n",
            "little_gan/discriminator/block_1/spectrally_normed_linear/spectral_normalizer/u  1x1024     float32  False      /job:localhost/replica:0/task:0/device:GPU:0\n",
            "little_gan/discriminator/block_1/spectrally_normed_linear/w                      1024x1024  float32  True       /job:localhost/replica:0/task:0/device:GPU:0\n",
            "little_gan/discriminator/outputs/b                                               1          float32  True       /job:localhost/replica:0/task:0/device:GPU:0\n",
            "little_gan/discriminator/outputs/spectral_normalizer/u                           1x1        float32  False      /job:localhost/replica:0/task:0/device:GPU:0\n",
            "little_gan/discriminator/outputs/w                                               1024x1     float32  True       /job:localhost/replica:0/task:0/device:GPU:0\n",
            "little_gan/generator/block_0/batch_norm/exponential_moving_average/average       1x1024     float32  False      /job:localhost/replica:0/task:0/device:GPU:0\n",
            "little_gan/generator/block_0/batch_norm/exponential_moving_average/average       1x1024     float32  False      /job:localhost/replica:0/task:0/device:GPU:0\n",
            "little_gan/generator/block_0/batch_norm/exponential_moving_average/counter                  int64    False      /job:localhost/replica:0/task:0/device:GPU:0\n",
            "little_gan/generator/block_0/batch_norm/exponential_moving_average/counter                  int64    False      /job:localhost/replica:0/task:0/device:GPU:0\n",
            "little_gan/generator/block_0/batch_norm/exponential_moving_average/hidden        1x1024     float32  False      /job:localhost/replica:0/task:0/device:GPU:0\n",
            "little_gan/generator/block_0/batch_norm/exponential_moving_average/hidden        1x1024     float32  False      /job:localhost/replica:0/task:0/device:GPU:0\n",
            "little_gan/generator/block_0/batch_norm/offset                                   1024       float32  True       /job:localhost/replica:0/task:0/device:GPU:0\n",
            "little_gan/generator/block_0/batch_norm/scale                                    1024       float32  True       /job:localhost/replica:0/task:0/device:GPU:0\n",
            "little_gan/generator/block_0/spectrally_normed_linear/b                          1024       float32  True       /job:localhost/replica:0/task:0/device:GPU:0\n",
            "little_gan/generator/block_0/spectrally_normed_linear/spectral_normalizer/u      1x1024     float32  False      /job:localhost/replica:0/task:0/device:GPU:0\n",
            "little_gan/generator/block_0/spectrally_normed_linear/w                          128x1024   float32  True       /job:localhost/replica:0/task:0/device:GPU:0\n",
            "little_gan/generator/block_1/batch_norm/exponential_moving_average/average       1x1024     float32  False      /job:localhost/replica:0/task:0/device:GPU:0\n",
            "little_gan/generator/block_1/batch_norm/exponential_moving_average/average       1x1024     float32  False      /job:localhost/replica:0/task:0/device:GPU:0\n",
            "little_gan/generator/block_1/batch_norm/exponential_moving_average/counter                  int64    False      /job:localhost/replica:0/task:0/device:GPU:0\n",
            "little_gan/generator/block_1/batch_norm/exponential_moving_average/counter                  int64    False      /job:localhost/replica:0/task:0/device:GPU:0\n",
            "little_gan/generator/block_1/batch_norm/exponential_moving_average/hidden        1x1024     float32  False      /job:localhost/replica:0/task:0/device:GPU:0\n",
            "little_gan/generator/block_1/batch_norm/exponential_moving_average/hidden        1x1024     float32  False      /job:localhost/replica:0/task:0/device:GPU:0\n",
            "little_gan/generator/block_1/batch_norm/offset                                   1024       float32  True       /job:localhost/replica:0/task:0/device:GPU:0\n",
            "little_gan/generator/block_1/batch_norm/scale                                    1024       float32  True       /job:localhost/replica:0/task:0/device:GPU:0\n",
            "little_gan/generator/block_1/spectrally_normed_linear/b                          1024       float32  True       /job:localhost/replica:0/task:0/device:GPU:0\n",
            "little_gan/generator/block_1/spectrally_normed_linear/spectral_normalizer/u      1x1024     float32  False      /job:localhost/replica:0/task:0/device:GPU:0\n",
            "little_gan/generator/block_1/spectrally_normed_linear/w                          1024x1024  float32  True       /job:localhost/replica:0/task:0/device:GPU:0\n",
            "little_gan/generator/outputs/b                                                   784        float32  True       /job:localhost/replica:0/task:0/device:GPU:0\n",
            "little_gan/generator/outputs/w                                                   1024x784   float32  True       /job:localhost/replica:0/task:0/device:GPU:0\n"
          ]
        }
      ],
      "source": [
        "print(snt.format_variables(gan.variables))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "V297xpzfobXK"
      },
      "source": [
        "## Training the model"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "WTrv-jn4pPSx"
      },
      "source": [
        "To train the model we need a loss function for both the discriminator and generator, as well as an optimizer.\n",
        "\n",
        "We'll optimize the discriminator via the \"hinge loss\", defined by the function `hinge_loss_disc`. The generator simply maximizes the discriminator's error via a linear loss given by `loss_gen`.\n",
        "\n",
        "For the optimizer, we'll use the \"Adam\" optimizer (`snt.optimizers.Adam`). To compute gradients we'll use a `tf.GradientTape` which allows us to selectively record gradients only for the computation we want to back propagate through.\n",
        "\n",
        "We'll put all of this together below in a Sonnet Module called `GANOptimizer`, which takes as input the GAN to be optimized:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "JgVCqok0usw-"
      },
      "outputs": [],
      "source": [
        "def hinge_loss_disc(preds_real, preds_gen):\n",
        "  loss_real = tf.reduce_mean(tf.nn.relu(1. - preds_real))\n",
        "  loss_gen = tf.reduce_mean(tf.nn.relu(1. + preds_gen))\n",
        "  return loss_real + loss_gen\n",
        "\n",
        "def loss_gen(preds_gen):\n",
        "  return -tf.reduce_mean(preds_gen)\n",
        "\n",
        "class GANOptimizer(snt.Module):\n",
        "\n",
        "  def __init__(self,\n",
        "               gan,\n",
        "               gen_batch_size=100,\n",
        "               disc_lr=2e-4,\n",
        "               gen_lr=5e-5,\n",
        "               loss_type='hinge',\n",
        "               num_epochs=100,\n",
        "               decay_lr_start_epoch=50,\n",
        "               decay_disc_lr=True,\n",
        "               decay_gen_lr=True,\n",
        "               name=None):\n",
        "    super().__init__(name=name)\n",
        "    self.gan = gan\n",
        "    self.gen_batch_size = gen_batch_size\n",
        "    self.init_disc_lr = disc_lr\n",
        "    self.init_gen_lr = gen_lr\n",
        "    self.disc_lr = tf.Variable(\n",
        "        disc_lr, trainable=False, name='disc_lr', dtype=tf.float32)\n",
        "    self.gen_lr = tf.Variable(\n",
        "        gen_lr, trainable=False, name='gen_lr', dtype=tf.float32)\n",
        "    self.disc_opt = snt.optimizers.Adam(learning_rate=self.disc_lr, beta1=0.)\n",
        "    self.gen_opt = snt.optimizers.Adam(learning_rate=self.gen_lr, beta1=0.)\n",
        "    self.num_epochs = tf.constant(num_epochs, dtype=tf.int32)\n",
        "    self.decay_lr_start_epoch = tf.constant(decay_lr_start_epoch, dtype=tf.int32)\n",
        "    self.decay_disc_lr = decay_disc_lr\n",
        "    self.decay_gen_lr = decay_gen_lr\n",
        "\n",
        "  def disc_step(self, images, labels, lr_mult=1.):\n",
        "    \"\"\"Updates the discriminator once on the given batch of (images, labels).\"\"\"\n",
        "    del labels\n",
        "    gan = self.gan\n",
        "    with tf.GradientTape() as tape:\n",
        "      gen_images = gan.generate(get_noise_batch(images.shape[0]))\n",
        "      preds_real = gan.discriminate(images)\n",
        "      preds_gen = gan.discriminate(gen_images)\n",
        "      loss = hinge_loss_disc(preds_real, preds_gen)\n",
        "    disc_params = gan.discriminator.trainable_variables\n",
        "    disc_grads = tape.gradient(loss, disc_params)\n",
        "    if self.decay_disc_lr:\n",
        "      self.disc_lr.assign(self.init_disc_lr * lr_mult)\n",
        "    self.disc_opt.apply(disc_grads, disc_params)\n",
        "    return loss\n",
        "\n",
        "  def gen_step(self, lr_mult=1.):\n",
        "    \"\"\"Updates the generator once.\"\"\"\n",
        "    gan = self.gan\n",
        "    noise = get_noise_batch(self.gen_batch_size)\n",
        "    with tf.GradientTape() as tape:\n",
        "      gen_images = gan.generate(noise)\n",
        "      preds_gen = gan.discriminate(gen_images)\n",
        "      loss = loss_gen(preds_gen)\n",
        "    gen_params = gan.generator.trainable_variables\n",
        "    gen_grads = tape.gradient(loss, gen_params)\n",
        "    if self.decay_gen_lr:\n",
        "      self.gen_lr.assign(self.init_gen_lr * lr_mult)\n",
        "    self.gen_opt.apply(gen_grads, gen_params)\n",
        "    return loss\n",
        "\n",
        "  def _get_lr_mult(self, epoch):\n",
        "    # Linear decay to 0.\n",
        "    decay_epoch = tf.cast(epoch - self.decay_lr_start_epoch, tf.float32)\n",
        "    if decay_epoch \u003c tf.constant(0, dtype=tf.float32):\n",
        "      return tf.constant(1., dtype=tf.float32)\n",
        "    num_decay_epochs = tf.cast(self.num_epochs - self.decay_lr_start_epoch,\n",
        "                               dtype=tf.float32)\n",
        "    return (num_decay_epochs - decay_epoch) / num_decay_epochs\n",
        "\n",
        "  def step(self, train_batches, epoch):\n",
        "    \"\"\"Updates the discriminator and generator weights.\n",
        "\n",
        "    The discriminator is updated `len(train_batches)` times and the generator is\n",
        "    updated once.\n",
        "\n",
        "    Args:\n",
        "      train_batches: list of batches, where each item is an (image, label)\n",
        "        tuple. The discriminator is updated on each of these batches.\n",
        "      epoch: the epoch number, used to decide the learning rate multiplier for\n",
        "        learning rate decay.\n",
        "\n",
        "    Returns:\n",
        "      loss: the generator loss.\n",
        "      lr_mult: the computed learning rate multiplier.\n",
        "    \"\"\"\n",
        "    lr_mult = self._get_lr_mult(epoch)\n",
        "    for train_batch in train_batches:\n",
        "      self.disc_step(*train_batch, lr_mult=lr_mult)\n",
        "    return self.gen_step(lr_mult=lr_mult), lr_mult"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 14,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 989
        },
        "colab_type": "code",
        "id": "UUkshshiK6Eq",
        "outputId": "6272bb79-7eff-46af-a913-b9e886772948"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\r  0%|          | 0/750000 [00:00\u003c?, ?images/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "WARNING:tensorflow:From /tensorflow-2.0.0-rc1/python3.6/tensorflow_core/python/ops/resource_variable_ops.py:1781: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "If using Keras pass *_constraint arguments to layers.\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "WARNING:tensorflow:From /tensorflow-2.0.0-rc1/python3.6/tensorflow_core/python/ops/resource_variable_ops.py:1781: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "If using Keras pass *_constraint arguments to layers.\n",
            "  4%|▍         | 31300/750000 [00:19\u003c01:15, 9470.95images/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\n",
            "Epoch = 1/25 (lr_mult = 1.00, loss = 0.016450781375169754) done.\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "  8%|▊         | 62500/750000 [00:21\u003c00:56, 12207.04images/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\n",
            "Epoch = 2/25 (lr_mult = 1.00, loss = 0.04956576228141785) done.\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            " 12%|█▏        | 92400/750000 [00:24\u003c00:53, 12197.78images/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\n",
            "Epoch = 3/25 (lr_mult = 1.00, loss = 0.10389317572116852) done.\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            " 16%|█▋        | 122200/750000 [00:26\u003c00:51, 12084.79images/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\n",
            "Epoch = 4/25 (lr_mult = 1.00, loss = 0.051943909376859665) done.\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            " 20%|██        | 151900/750000 [00:28\u003c00:49, 12153.37images/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\n",
            "Epoch = 5/25 (lr_mult = 1.00, loss = -0.21330127120018005) done.\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            " 24%|██▍       | 181800/750000 [00:31\u003c00:47, 11858.25images/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\n",
            "Epoch = 6/25 (lr_mult = 1.00, loss = 0.15133343636989594) done.\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            " 28%|██▊       | 211500/750000 [00:33\u003c00:44, 12015.89images/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\n",
            "Epoch = 7/25 (lr_mult = 1.00, loss = 0.2119884043931961) done.\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            " 32%|███▏      | 242500/750000 [00:36\u003c00:41, 12089.66images/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\n",
            "Epoch = 8/25 (lr_mult = 1.00, loss = 0.25304022431373596) done.\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            " 36%|███▋      | 272400/750000 [00:38\u003c00:39, 11972.72images/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\n",
            "Epoch = 9/25 (lr_mult = 1.00, loss = 0.31501442193984985) done.\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            " 40%|████      | 302100/750000 [00:41\u003c00:38, 11631.19images/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\n",
            "Epoch = 10/25 (lr_mult = 1.00, loss = 0.3819904327392578) done.\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            " 44%|████▍     | 331900/750000 [00:43\u003c00:35, 11762.71images/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\n",
            "Epoch = 11/25 (lr_mult = 1.00, loss = 0.359415739774704) done.\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            " 48%|████▊     | 361700/750000 [00:46\u003c00:33, 11701.33images/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\n",
            "Epoch = 12/25 (lr_mult = 1.00, loss = 0.3620455861091614) done.\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            " 52%|█████▏    | 391300/750000 [00:48\u003c00:30, 11574.55images/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\n",
            "Epoch = 13/25 (lr_mult = 1.00, loss = 0.34004005789756775) done.\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            " 56%|█████▋    | 422000/750000 [00:51\u003c00:27, 11761.06images/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\n",
            "Epoch = 14/25 (lr_mult = 1.00, loss = 0.4108557105064392) done.\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            " 60%|██████    | 451400/750000 [00:53\u003c00:25, 11703.31images/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\n",
            "Epoch = 15/25 (lr_mult = 1.00, loss = 0.4520989954471588) done.\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            " 64%|██████▍   | 481900/750000 [00:56\u003c00:23, 11577.90images/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\n",
            "Epoch = 16/25 (lr_mult = 1.00, loss = 0.46710577607154846) done.\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            " 68%|██████▊   | 511200/750000 [00:58\u003c00:20, 11707.29images/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\n",
            "Epoch = 17/25 (lr_mult = 1.00, loss = 0.4239480197429657) done.\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            " 72%|███████▏  | 541700/750000 [01:01\u003c00:17, 12108.87images/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\n",
            "Epoch = 18/25 (lr_mult = 1.00, loss = 0.40935561060905457) done.\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            " 76%|███████▌  | 571500/750000 [01:03\u003c00:15, 11730.40images/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\n",
            "Epoch = 19/25 (lr_mult = 1.00, loss = 0.46318596601486206) done.\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            " 80%|████████  | 601400/750000 [01:06\u003c00:12, 12025.96images/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\n",
            "Epoch = 20/25 (lr_mult = 1.00, loss = 0.45752182602882385) done.\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            " 84%|████████▍ | 632400/750000 [01:08\u003c00:09, 11889.06images/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\n",
            "Epoch = 21/25 (lr_mult = 1.00, loss = 0.3791620135307312) done.\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            " 88%|████████▊ | 662100/750000 [01:11\u003c00:07, 11857.17images/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\n",
            "Epoch = 22/25 (lr_mult = 1.00, loss = 0.2888197600841522) done.\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            " 92%|█████████▏| 691900/750000 [01:13\u003c00:04, 12035.37images/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\n",
            "Epoch = 23/25 (lr_mult = 1.00, loss = 0.4309738874435425) done.\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            " 96%|█████████▌| 721800/750000 [01:16\u003c00:02, 11965.87images/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\n",
            "Epoch = 24/25 (lr_mult = 1.00, loss = 0.28471168875694275) done.\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 750000/750000 [01:18\u003c00:00, 9546.01images/s] "
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Epoch = 25/25 (lr_mult = 1.00, loss = 0.1431596875190735) done.\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        }
      ],
      "source": [
        "import tqdm\n",
        "\n",
        "num_epochs = 25\n",
        "num_disc_steps = 2\n",
        "\n",
        "# We'll turn the step function which updates our models into a tf.function using\n",
        "# autograph. This makes training much faster. If debugging, you can turn this\n",
        "# off by setting `debug = True`.\n",
        "debug = False\n",
        "\n",
        "optimizer = GANOptimizer(gan, num_epochs=num_epochs)\n",
        "step = optimizer.step\n",
        "if not debug:\n",
        "  step = tf.function(step)\n",
        "\n",
        "train_dataset = iter(mnist_shuffled)\n",
        "num_examples = mnist_train_info.splits['train'].num_examples\n",
        "total_batch_size_per_step = batch_size * num_disc_steps\n",
        "steps_per_epoch = num_examples // total_batch_size_per_step\n",
        "\n",
        "steps_with_progress = tqdm.tqdm(range(num_epochs * steps_per_epoch),\n",
        "                                unit='images', unit_scale=batch_size,\n",
        "                                position=0)\n",
        "\n",
        "for step_num in steps_with_progress:\n",
        "  epoch = tf.constant(int(step_num / steps_per_epoch))\n",
        "  train_batches = [train_dataset.next() for _ in range(num_disc_steps)]\n",
        "  loss, lr_mult = step(train_batches, epoch)\n",
        "\n",
        "  if step_num and (step_num % steps_per_epoch == 0):\n",
        "    tqdm.tqdm.write(\n",
        "        '\\nEpoch = {}/{} (lr_mult = {:0.02f}, loss = {}) done.'.format(\n",
        "            epoch.numpy(), num_epochs, lr_mult.numpy(), loss.numpy()))\n",
        "\n",
        "print('Epoch = {}/{} (lr_mult = {:0.02f}, loss = {}) done.'.format(\n",
        "    num_epochs, num_epochs, lr_mult.numpy(), loss.numpy()))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "2K0_eoR8og-G"
      },
      "source": [
        "## Evaluating the model"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "Cm_9RMJopgWc"
      },
      "source": [
        "Having trained our little GAN, let's check what its generated images look like now."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 15,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 99
        },
        "colab_type": "code",
        "id": "PM7IPcOeXtxH",
        "outputId": "4215ea70-f36a-43bd-fa9b-b232cea584ef"
      },
      "outputs": [
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlMAAABSCAYAAABwglFkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJztnXd4XNWZ/z/n3hmVUZcsy2q2bMsd\nTA89QMgGAkmAhADZEDYkgV96T5bsZtOWlE1I2fSwoaRCSAIxSQgdQrWxwQ1jLHdblmzLsnqbmXvP\n74/33NFYFrZslRmNz+d59Ei6unN133vKPef7vuc9SmuNxWKxWCwWi+XocFJ9AxaLxWKxWCyTGTuY\nslgsFovFYhkFdjBlsVgsFovFMgrsYMpisVgsFotlFNjBlMVisVgsFssosIMpi8VisVgsllFgB1MW\ni8VisVgso2BUgyml1MVKqQ1KqU1KqZvG6qbSCWvj5CfT7QNrY6aQ6TZmun1gbTxm0Vof1RfgApuB\nWUAWsBpYeLTXS8cva+Pk/8p0+6yNqb83a6O1z9qYWTYezddolKnXAZu01lu01lHgbuCyUVwvHbE2\nTn4y3T6wNmYKmW5jptsH1sZjltAoPlsN7Ez6vRE4fehJSqkbgRsBXNxTIhSO4l9OLDnk4RGjUJXq\nfnoAruMYtDFT7DOHuoBfDz0vU2w8luspZL6NmWKfOWTbItbGdKefHqJ6QB3uvNEMpkaE1vpW4FaA\nQlWqT1cXjve/HDP26EZa2c1CdSrL9GPEiA57XqbbmCn2ATyq/7RvuPMyxcZjuZ5C5tuYKfaBbYtY\nG8cepUCP7X7Dy/RjIzpvNIOpXUBt0u815ljGkE0u/fQlH0qNjY4Lvjcul04bG8eJYezLIoPsg8wv\nQ7A2ZgK2LWYG6WSjCmcB0P/GEwBovDZO4dM5AFQ+1ASA17QbAD0wMK73MpqYqeXAHKXUTKVUFnAN\ncP/Y3FZ6UEgJfXTTp3vQaLA2TjqS7fO1D1BKBtkHmV+GYG3MBGxbzAyOBRuPhqNWprTWcaXUR4GH\nkOj+27XW68bszsYTZdyfKmksOYzy4yiHefpEVvI0ffQC3DOhNqrDumlHTcptHGeS7TMNf38m2Qdp\nWoam7qqsLJz8PAB0v8wM/d5eOecI5Pi0tHGMyXQbbVvMDNLJRpWTDcCuC2Qoc8PiJ3m4YgEAW8tq\nAKj7aScAXhorU2itH9Baz9Vaz9Zaf32sbiqdmKIqOUtdTD5FWBsnJ4F9Z6s3A+xO9f2MB5lehmBt\nzARsW8wMjgUbj5RxD0BPF9w5swDY+e0cfrT4bgA6ffGtfupv1zHv1lYA9A7xs/o9PSm4S4OZ1Ycq\nppp76UVHJcgv+D7WQXbjRqCujfR+k9W4yWJjgFIo1wXAKSuVY0UFAPTNKiXUHUucBxBu2IW/vx0A\n7RlldJxi48YFR2xVjtjT+5aTaVksXcpXr/0dAIuydtOv5bx3Lb0BgNnfkjrsr14/obc75phydIuS\nVin5Ume9btN/aD+967EpQyc3J3FIhaUMVXERAN6UQtx9MrvXxi6/q3vwGqbuamP7pKrDmYrjDpap\nqX9+/8Brlk0QewSg47EDPpeuOAUFtFxzHACnnf0qAKdHNrO9qAyA3i1VAPjdE/Mut9vJWCwWi8Vi\nsYyCY0aZanzbNAD+ccq3qXQjAAxoUQXuuvxHXJ39EQDmfWJ7am4QUNni/23+0CkAdM2WWUTObpe6\nv4hyxh5ZTey3dwzOBCWY80BSOatIVmgKRJnR06eh4nKferM8Yz8aS3zEyQrL9/IpAPQsrmSgUK5R\nsnyPnL9TVMPxXpVxxASz+7wIarrMhgam5QOw419kxufl+bgyYcKLi6JRU5FHW6/YG19VDEDdLasB\nE1eU5jPDxCzXkW4kf2MHOy8Tu0/JlsU9daFI4vT3H/ccAPcdL0uki9eFBhW5dLc1Kc4yUOIC9dGv\nKmfrlaJORatEdZv3w345vWFbalTuZEV4iPrkTCllz5sknqTbrMeeda60yY/UPk652wVAuSPtbL+f\nRYsn7fjWptcDsOFB6aPKV8bIWyXpBnU8KEsfb1/reFlmGY5AJV0wB4CtV03hHZc/DcA/dkoMkf/Q\nFKoekD4UX/rifedWA9B2aQ+Fj0hs49RnWgDQjc2p9dC8Bm6htLW2Sxcy/T2bAPhIpaQvKHYG2NIl\nHW37PNGKSkzMptc2fAqHsSLzB1OmknXNjgNQ7IRwTeB5tjG/yw9Td7905sq81HVsfB/8UJxIhIab\nFwNwzxU/AKA2JIONB3pm8v2eKwEoaZCKlLdyJ7pHAnn9YHARyO3x+ITddzIqZNwDubn0XCgNuPFy\nuae3Hb+aVV88CYDcHWbg5Eg5OCXFNF1RB8C/fvAhAN5VeDcVbi4A3b7Y90Cv9Py/vfqiQRdRigeN\nMOgWcUqL0QNSb7Kb5YU057ttcu7AAMq4g2I10tg76qfh18o1yteZMgvsSffBRRLBoF67is+e/jAA\nNSGZGLjKwTOD/XPyNgDw4L7z5YOuCymqq0eM6TPckiJUgQwYvSlSnjtugs8sWgLAgC91+w+zLgag\nYL03/EKS8SrfoE6ayYz2NW6JuOuix9cBsO0GzYdPkHb2loK1ANS4ct/ZKoSsJ4IBLROBctdnXrgD\ngDkz7gNg1Xtl0rDyqhn8bumZAJSulM9Ne7gJZdyAaTfxORRHGpKQDgQD5eNkEHXir14B4NdTnqfI\nkcHzZ8qWAXBv/Sy+vkASlc+eL4Oq39d/B4BKN4snTpX6fNPP3wdA7Z1t0CeTgsSEPYXPJhAb2i5d\nCEDlhzZzY9VTAMS09MGf3vw2Om6XiULtVhkIJurgOKYYAuvms1gsFovFYhkVGa9MBTO0eXdIkrEN\nb3ZYnCWjUx8ZbW8cqKVzhjyK7CcmeKZsZkMbvrmYl9/xQwAijswIO8woekXXTLLbZEYwUCz2cHIt\neevE/eXkiYITBM/r7u4JnUEkFCnzXc+vY/91Miu496TbAYji8OxHZBFAbpPMHHpmiuug7t9f5cdV\n3wZgunELuSo/cf0S45a9Jl/k52/8l2LG9fJZr7NznKwaAeYZB4sC4jubEi6gYYNxzb2qRnGBla6J\nUGZULcyz8/oOSIY3KQiUuf0nlLAo+8DcfTHtJdpZuy9ltu84UUFqN1fjbdoqJ6arGmBm/qEqCRPQ\n/f0J18hpH38JgFsrHiPHtOMPbr0cgH2L5XOFj+eCcWcHfRHaHzf1OKFIBe5T5aBypX/onyLP/ZpF\nz1GTJW64AmX6FS33s88fVOSDmfaAhrA68JiHWUShPC47TZ7DktCJAJS/VAiNzWNr2NEw3AIBEIXY\nlEl8hizy6aiPsPd0eRala8TKwq3yLLqmZ1G2Wtqu0yh9kNfSmhaB9qFaUQjDPxIF/IvlKwDIVrmJ\nc7qMqrSyewZnnSTq8I3TngQG3fCucjgtW+pEdodpi8WFCYUx1V4PAMcowl3vFNX/i1VP81TXPACe\n+0/ZzSby/CaKe1cCg/2yH6RAGi4cZizvb1yvbrFYLBaLxZLhZK4yFSzbNrMy1S2+39/vP4OCKRKY\nl2NmZf/z1KUsvF9myPHoxMZKhWbOAOB7l/w2oUgFPNNfAsDWt5dT1iKjbYLZreNAqQQtx0pldqHL\nxW/uPrUa9MTNmoLZSuDTjpZkc/tJPwNgUZZUsQEdo7pA4i7aq6cD0DZX/va1qc9QZWJsghlyGBeH\nA2NN4ohNZfm9YGLb0oJEnJM3ssmPGowVi02XAHTnxVcPvNZkIJj5l0o9zX1PMydliyIZQsqzT0d5\nKSqxGy1xUQi8M4yauESSekL6pfwIVNYgyFwXyaz41c/V8qELJS7skvyXAShQDi9FRXXb2FoOQM2T\n0t/oaDQR+J1QpsKhRAJTbRSSsYrRPFg58NEx+R8DhVLvlrfOoKFbFJknsyW28fmmOgB6erMJr5WA\n3f6FopJev/h5FuSI6r20ezYAa9tFESnO7mN3j5RrTqOUpdPbheelSLUJ6mRxMe1vEtWi/pMSR/Th\niicAiDgxek1sW7krNnb5YaJGW+i9ROpusdNnzo/z67YzAFj+QYn7VK37x1voODxK4ZVLPNz5Zc8D\n4Br7fTTrolIXvrf7EgCeXjWfunrxZswPB4Hl8n6MaY9v7D0PgNJ1Eour9wxunZhQ2lNBEMtWXGh+\nlXtpiRfy8E/OBqDswRcA8IZTCyfoXThpB1NOnjR4lZszmPskaXWYayL4VcQEMc+WDn9qVlci781H\nNr0TgLIVLn5HatxFvXOl831T7n5kqypo86Qy/+T0NwDgtTYOfiAIMA2FwayeUaaity6STnvaqny8\n9o5xv/ehBC+GaKHLNDcIPJXnvz2u2f99GTjmL5MVGNn7xF3y+fPegetIz7S7WcqpZEoXH58rnd9p\nObLS6J4O2SC1ac005oWNO2wyBY2aAf6Wb7wOgB9ceQc/2P4v8qerTVBz8qAi3W0zg8KWi2YC8OVZ\nd5CjpEvxJcM1e7w4X9okrq8d68VVlrdLPqdz4wnp3u8Q6X6iF34Mi+PilEg9jM2VOto1Q16wFA/e\nX1hJnd0eD/GNrZcCUH2zaZ/rZXCsY/HBwZSxFWfQITDuAdpaJ1w0kRb5HvVd1rdUABBbKy/jymel\n7VY1tNA7T/qhgc1i85q6avJdGRwWhaTdNeyQsszZlE1ui5R11SYzQNy6c+JdYMn9IuDX13DuTUsB\n+NrU5QCEEoH1Di8MyHk/b5U+5d51J+I0Szldf/HjAHy8VFbWhlUWMfPO6KmR/ixvWepdfMp16amV\n91yRGRS2eFIGKwam8acWsW31XyRgO19DxWJpZ/2mTwkmqD9rn8PKL58MQO4L4rb1PS+t+h6/WGwN\nOTJIfLh1EVOXyLtk2EHUBGPdfBaLxWKxWCyjYPIoU2ZWr0+XjKfbLpJRasXyGLmPrT3gVOUolFkO\nHORTqbpW3HhvLhg8d9NWmZ0t+EvDYODvRI3EjT39n9gPcICL75KX3wNAYevmgz9n1ADlOolcIVvf\nZlyZM0Sh69o7n7z7XwQmNmAwyJzrxDRdvtjnaXmuVyz9MLMfFrdI8KzdmNxb+ftz8Y2SVuRJzhqV\nk813PirpIN7/7gcBuKdBZk6Vz/uJgO3E/ooT6NY8KpRi828k9cX6838MQFi5zK+/C4CPh8XWRP1L\nXk5v6ko6BLwCiXtz8sS9vH+RHJ4TbqXLuD46jFr6ue1X0LhGVIxpIhDgZSfZWCYKEClQUl8L5ShU\nyLTPcmmXRoRi2gNZ/NS7AICf9b0RgJwml9rHTCqAlbJFWaIX0T7aExUkKFHd0YlOlPM4z2eVSrgs\nWxfJ9/qsfjAiWWy1uCfdATEwNq2YaIHcU+vxcsdXFG+joVfK8MF1UtiFL4pqVdIQJdRjXPN7RN33\nYykIUk642sWO3qpc3lYkCkuXCaoP6uQXdl5G9/XyfvC37gCg3l9DqEK8BM+cJq7Mz5WJe3Cf18eS\njccDULfEBDeniWKz+3STssMs8lgdlbCBn+88jw0bxRVbaMTPzsVRLp8i9x8xbfhzzWcBsOm6meQ2\nyPNKZZD5ofDypB19Yp642b/6xOXM3bd85BdQalzf71aZslgsFovFYhkFk0OZUordH5Olj//90TsB\n+O0eSRTXfVcJXhArZWYlKpRN7wKZSb3pw88CcEa++Fb3exGe65FA7QXfluWk3v72CfcNB8GoX5zz\n98SxILlh0ZdMQHniZIVjgruDXbK159NzoiSx9KslnuHLJz4AwFf6L2NOk/jJnVUNck5//zhZkoR5\nhvnrW3mkRwJbg1iDWd/VktU7Ca9L/Pd0dR2syPRDdrscW9Ul6mJBRGyI5uUnBSynOgp0ZIRm1PLk\nuT8CIJyU9uGDm94FgLO78cAPHBAzlWY2BmVlAptz98p9/nTfeYmYmuf2SRqMxqdrKZDV5LTNl+/5\nO+Xzza8voXiTuUaTBMbqoE6kgkTcTSixwKHgVekjVJtRXaaW0F0japoJUyHUr4lH5Pzw0NQYWuOb\nxIfBjN/JzQETHBykNfHaxy9WrH+htJ/TLhdV/sopy9kdk8Ur3znhCgBqJEyIaHEW7fXSZqefKSpx\nkdvHxk5RbYqWS/9Tul7kjpwNzWBUPL/FZD1Ph/qq4KU+ieUrd0VhuvivnwFg/lc24e3bcvBnTL3+\nyax7AHAYzN4//fsmzi8dYvoCXJfIHqlvs7P2AlAVkvbzzsoXWZonO3y8WCnl/7fFdyY+umxAEgdv\nukb+5m3ZnD7K9xAcs4is5wuiXp+XK2WXsyd0yPd2sCBKBUH50di4ejCsMmWxWCwWi8UyCtJbmTIj\nyo53n85jn5W098Gc57NPzQVgdtPLgzOhIJ6oppKK/5J4ow+XyX5gHSaG59VoBb//vewNNr1D1KpU\njMiDbWvaPbNvkO5lc9zEEpkd2r2k7VmCVYnB9huqvCyRaPTfTxF165I8WfWWd9YfuWu2rBjru8bs\nH7araVztARKxPX0zS/jlBlmy2tsts4MF+1o5yBOfPKsIUlkE+57NrKXtBCmX1XtkRVX/WplNz3y5\nEx2svkyT2IXDEasqSdTBGnNsn9eDe4nMKIeLwUgspTekWyxDcD/Vj8uM8aWNJxPulDJrOUnK3cmC\njrnSPi8+axUACyKS0PEvzSfQ+Xsp21yjco13XMMhSfq/ulfUJD9QzMyKOLevj8geibc5/mOi9JxW\nuJXv/lm26Zi13CTQ7e5Ouq7Zk9Ks3NNKQdisOusbX8VYhcLsPVXK4mtT/wnAnHAfz5ntN7IWi3qx\nU0nbitZEufmsPwBQ7kob2xKdyvYWsxp6tzyH7J1tCZuUSTOQSNrreSnbJihQBHNaotzaIH3QHyMS\nazl1qVEoOoZRPx0Xdbe8P6Yn7ScJcM5TH6N+2ZrxuuWjx/MwGTkSaRwqXLHhsvzNnBuRd2BOpTyT\nHt/h/q4TAPjzLRLvV7JFUgqkqyoF4JjUK39c+GsApphtxvqnxYdd8exEjGdnniiTXqAab28h3rRb\nThoHe9N6MBWqkY72tpu/T4kjD7DVl0pTtjbIMO0nXGB6kQQOVv90G9+vlo0PHZNu4KdmCeyfXj2J\nun+aHBtJqRQmmiCNwG92Sf6Sc+v/kHCNxSuks3Y7pdGrcDix51uwzJlwiI758gzOj2yUQ2YweVZO\nE5uKZWD191MlWDZ3PAdTQUCycUF6uQ6xmAwEwtnSqfbOLSen2WxYPMQt6+TmDkqyRdI7bL+igki5\nvKTzc+QllGWSZTs7mvG9NHAljAQzSAxv3cOrUVnwMCMkA6grb/wk2QOvHUB5yI2s04DE4G6lpAHI\nXTlYB2pWmn203jyPgVKpH1eVSsdd6oq7t7y2k1vca4ZcNPWDYz8ag30mx85Q97Pr8t7/+CsANxSJ\nG8xH82fjEgsWhRxgx5BrOCXFaD8ISTCZ78c6RULgspw3i4uvkRxEC8MycAsrlzlh8b1eNUsCkucu\nlAHuidlNxLT0Iz1mwOUozayp4sLrDEtoQbRGBl/ZfQN45fKz02j6poGB1KX1MC/JUMMu8u+R98G+\nerOJ+HJpd772D9ov0Z0zk7vrfys/K0mRsNeT98S8T+5Ii6X3w5FjqmmQkd43zzusHIpNupktMXl3\nLu+bxW3rJDxmzkPiKksMeVM5iTkUSrH+ZgnZqQzlH/Cnb17wR34VkXCWYFLiFhXSe2Y9ALGPS509\nr0ImPQ/cfg5Vv5FJznikDrJuPovFYrFYLJZRkJ7KlJnNb/6AZMquD4dwjepSZFIIlNwoS1o3XDqX\n02duA+DrNb8AoCaUC4i0tzUuI9Z7HhXJt+pZj9CWbeb/DLOb+wQRpBHgJnHDffF/L2FXj8ygcnaL\n9O73yAze1xptlhsHrh+npY3pi0wQqMkIGzZJ6Qocxa4BuVZkh9k5ezxnHnpQJQRwoppQSGZy88pl\nNrj2gnqmRURizt0tZdJ4obg4s9uhb6pcI1oi1/jUG/6OY9ajFxgJ+8uvezsA5UuS9h5LcxJ7peVH\neLlPHHxPdIgCmfPISl6zRJJnzuk4Y0wmadaeWGRg0l+ULt2N8mVmeaP3/wCY9Tppuw07plFmPqrT\nQWk8xDMPynHPnVO4oehRgESfhPbZ+oIoNjN7hiwkGOYahELEayUAONRgzh+r9BdDklfGS3K5tEiS\nT0YcORbTHl2+9KMX5Etw9q64CapH45laud8TJWDAD1OcbTKFv1fa8+5eUTt6uysof8SkSQiZzP5b\nPLz9qQ3U9lr3U3y/1MUSo5b6JrmzTk5GaZ5Xf20RuerAHSguXvk+AMpbGybilo+K8pVi00Ndkrqh\nKCQ2d8QjhE1yyxc7JFly3HcofFT6XEzYiGPCTbTnD7qj0ymcQDl85nUPD/unk3MaufljsoCnYIdR\nej2IXyeK1I/nibu6zDGq7/tgxYsmg/3zxm07hn2rVaYsFovFYrFYRkFaKlNB8rTISTLCdJLGfMHP\nt86W5asVc3MJqyBQd9CnOqBF+XmyV9IgzLpPRuzu6k34QSBoKvcbMiNid6PMTNfcdRx9FXJsdp4k\n8gxmCNrXicBsFTYxFjOncVmV7DE4xah1MbM1wB7P58V9MlMuapSZpDee6kaw9NQ818gLWxi4UJ77\n6XMlTubCt73KivPrAHhylSgztTMlTmNKbjcvPyN+7qCom6NFvLdUYj2qzIx+/7kyQ3mo+gwcs7dZ\nIoA33WIa1IHlpeIedz5/DgDZe+TYDH/Za34O5aRtrNRICPbcQ2uKH5NA2JJ/SjnGa6V9V9eEKHhW\n/pZ2liaXA9D/LzKjfeHkXwwqUoaGWD+zvmSS5B7ymiYWaXElkQ37DvzbGNXf4LkH35vOzU0ovMls\njsnefF98UtTeKdUSQ1Ka20vDDonty9sgis78SxtYVCht9Z3lKwA4LWcwBnPVmXKtTz4oyYYXfDcf\n2kxMSqqS6foefo+JjQ3U0uH6QHNs95nZiS2QmuMSVzPteokrG9e+c5S4qyVe9pk3mGUtU0XxjJXl\nsec0UZ8WXynqY2NPMe0XSH8ZaZEyM/Hr6P1teJ1JiybSBOUo5mcPH+9b6sDl18g78C93nwtAuBva\nmyXmuGmuqK212RJ0/u6SZfz9ddIHV75gPAZjqMKl5WAqqODdvdKY93h95JmOaEtcbrnU9Gcx7SUG\nU8EAqsOPssq4uW77pqyyKVkunZ2fTnlCAN/k1al8qp3Ob0lF395tMkYvk8FhuKOfvkqRZ/fPF/v7\nTu7lorxXzFXkWODme7ynHm6TF5bulaDvcQ0wHHrdeJyI2X+tOSoVuy4nRp/JBH38AnHzNP9KVlus\nmwk1T0vZhdtlkPTKcZVEyoJgSrHrjFx58d5x4SVU95ts67ukoeh4POEKTQxCUtkJBpsZm4mBzgpT\nslrsyNtj9lR0XfTQl2giw72LjqfdEAMVCg1OQoYZAAQB1R1vl8HH7jfGmfczqdfOJgnSDu2QAX7h\nnqzBFW7x1C0GSZD07FWOrAjqfqMEuN70nV8BHDCQipmBwqff9j507NXDXt6tlna97/gwU2Py0stZ\nObYvsCD/WrAfYDyi6fWlH+3yZXCxbKCMW/73agAW/l3KRBtXGOSwwJeVekH+qPZl07nbTI5Ouvo3\nAFS7gyveSnNl8nfFWbKY4sUHTiFiAnxTsUfoQYwgF9GbL19KsyeuzGs/9mkAcltfGP97GwXa89DB\nzh3B91Ypi3BZKeVZdQC8cr4MjkOujzcgZbrzMpPTcIZkSa96KITqkWsc1CelEuVwT6usTD+/WnJG\nDu776fC7F2QBl1su9oS7HMJ75T2zob8SgHlhIyigKL10l1z3J2PvlLNuPovFYrFYLJZRkJbKlN8l\ns7X6L8rs560XfR4nZgKUi0V+711sgpJP/SvVYZlJPdYpe5+1xvJY2lQHQNWfJPgy3RSpgCCQur86\nj5Z2KY6qC0TWXPB2kdY749ns6S2U8x+RoPzjqpupNQGfwb5+vWYPqmK3l8INZkaYikDtUIjy1aI4\nPJwnswr/+C7CK0RULtwms4jyf4rSVB6P4xm3gDtH1KpND8zmpQ+IFH1Gjkmhjcy2T7jqZZ6ZKa7C\nuXfIMadxL36nyT1llJOUZCsO3HtBsLH5PV6WR/8U+XnKKuOaTF4AMWSptva8pH3bUqi0Bcv580Ul\ndaaU4heZINaNkn7DT3KjOHXiXr7g85Lf7R3FK7gqWwLPZ94mS9UHimXmWLh0O1465AszNrqFYqPK\ny6P3eEnL0n6dKMdvzA1yE4UTOxXMffhGAOZtePnQ1zfB5e2nyUzZD0N/qbT17CDj+5gFoJs6E5Y+\noeikffQYZeqBHglE/trfrmTevdL2vFaTLyrYVzM7Gxyj0Jng5PCOJmZ2Sbv87XmiBFw66yH5m3LJ\nN6kE3lkiytSj888g8sKBOdLSDlPmG34k74zfTv0B124UtS7yd7NHXWru7DUJVN9ATdMDA4MLcYL2\nY+qR39WNHzbq+BJZ5NRyVozj6yWs5MoKcdc+ME/sb9peT95Oee+kU5Z37Xk8c5+kNdr2wUcA6De7\narz/lfcw9Rl5JpG9ZkeBaIzOM6V9/nWXBOVftUDSf8S0w+52eY/OzDe5HNvGzlarTFksFovFYrGM\ngsMqU0qpWuDXQAUyWL9Va/2/SqlS4A9AHbANuEpr3TYWNxXMdNU28edX3LorMYt3iiUGR1fK7thf\n/cxbqTTJHbv6ZcTev7aY+p+YpGRD9oMbjn7dyzqWE6UfUFQzk+lqDjEdZS1L6aOXKP0opUrGysYE\nZkaR+9Sr5M2UkfT0d28DYHG+2L8vVsCOLpldxCNy/pmlWxJLeYOZsm8UjC+tfCv1TeIb9gYGxD79\nwiHtyyWCHqMwYL+9g6xWUQ6z2qVMpv7AIdQhCwpUh0mctl9SQCTHy/imzMtXl3DLlosA+MRMScBa\nHZJH/4lpj3LyhRJ39bPONwNQ8kSMzU/dz4DfC76mJjyXWqYfZCMwvlPmxAxR6qu3S9RFt6WVGZ2i\ndqjdJrA1Fh9UJIYEmztZ4cHkpoYJr6dK4c6uA2DDRyTu4urzn+MP604BYN7XJAaITdvM+dD8JlFf\nlkz9k9hBiH+c+2MAPl1zJQAtS0TlyHug7SBFKiVtMYiVyhGFpX9+JW1zRJV5+yxRtmNJgdR/65F4\np4VflXKMR6MHZv+GA5beh6rN2bp+AAATdUlEQVTlmXTOdIl1trHvB7+hsV8UqRq/jumh+cS8vrFp\ni0E9MvVv//oy2meJwr9k74kA1N/VNbi4xRuM35Pf/UT28oQtnoezU+JOtrSVHfA8Bhf/QKsviqWz\nYz8v7F9CVPeD1q/Z3zDebfEQtF0nCtvyN98CQIunUB8X5dyP7zrs5ye8njouaqEs0omWy3PO2bQX\nf5/ESAXlp0rk/YjrkrvV/Fst8cP+ijALTpYY0xNzRKGaViHvzq/79YMxp6mycTi0T8Vy8XLccbUk\nHJ2ebRLIPjeVmY+LOq7N3rPN18xjQY1kd/7qjCUAuEb0v6ftNNwXCxLXHWtG4uaLA5/RWr+klCoA\nXlRKPQK8F3hMa/0tpdRNwE3Av4/5HU4ACsUcFlOoSojrGC/wGKW6gma2UcpU6tR8ntJ/xyM+KW0c\niX3b9KvsYGOqb/WoUcphXuHZFIXLiXZ3sLTnfkp08UE27mfvtFTf69GS6fUUjgEbHZc5My6mZKCQ\nuD/A85tvp9SppImGjGmLKJd5WadS6JYR7ek4sAxVBXXMs21xEnAs2DiWHHYwpbVuBprNz11KqfVA\nNXAZcL457VfAk4zVAzWzusSyRcfFzRdf58BxEpOx402iypw5ez0vPiwrbrJMmFD9fY34R7CKJFvl\nko0sIw2pMBFdwAB9tNDEKZwHQJgsovRdzjhVGr+ri8q7NwDQfZWoOXOzZBaxK1rCvm6zh59ZcX5F\n4SpcFezrJ6PsOzrmAVBze9bg/lNak00O2Sau4bXsq2QGmwlWB44O7WvcdrO8+DmjKPbFBhWpPWY/\nuuRlqYlkg2ZlYmeU/jtFDfn5B+Qe31klKzIvjDTwhjxZPfXw2VL2XS/VUlA2Ew2EOrvIU4XD2riJ\nl0vGxMjDEKyqStTlgQFYJ+WbUEKSYqa0b+IbglVVvj8Yd6XlvGw9sfXUyc6mv04U0Xsu/yEABU6M\nC89cB8B/z5akhrlmRaWqnsYjX5A9NMOmbgLUuKLyLCoSlc5/SGaHw22hkpK2mNjb08RjTg3TNVuO\nvT5f6lmvUWI8P86Pt5stmrLFrlBNNfFqeU6hbWbLpKD/CYfBpMeovUMGSLqrGz/ejgNEdAH9sc7R\nt8WhMXcmQWVWm8N9e2Rl5boGWT4/z+lHmb0Bg6SNqsDM2OPxRN0NVlriuolUC+1tUq4DWtpuhKzE\nKuptUembp+wO48Ty8WMDB5ehPg/UxLbFobiFhfzyq98HoMDEm75l7bsofnXTiK8x0fVUhUM0XSB1\nrHOhPG8Vr8LpE7U70mz6D9PtlG6IkrtJ0m9k7xUvgT83i5nZLebepH53+WJDPEcdlAg5Hd6LALmv\nSv/ybMssAP7RL/GypesH77fzPFHtTr52DR+ueByA2pA8p8d7pV7+4aFzmHuXeD480z7GkiMKQFdK\n1QEnAcuACjPQAtiNuAFHj+MetHGxk5tD54WysfHbvyJBaBVh6azavQjPVsjfIs3y8onOKMM1+8Ad\nKX26hy7aKaKUKANkK6lMSvY+GhsbX4PA7bVyk3R+DeUycVuQ00Rvk+TH0IVSgbr8cCLgfL3xBv3i\n15cCUPvUS68ZcP9a9mWRgx6rkEvfS3TmrpGOdTx+YAbiZJRKZGx2por71tOQv1Ok271dEhjc7cmA\nsMINsSkmL49trdLBTPE1Okeu0Tuwny5vP0WccpCNTNSii0PktCFIYRQKJVInJF6GcXk2urs78QyH\ny4UyEfVUx+O0nCyDuypX6tMUN5cCJW2v6TpzrFDc0zfd/GumuHkHXWdlVB756huOk+uuXzei/z9h\nbTEY8JpBRH+pg58njeqBdsnaXztF8tlsiZXxrhoJtP7FLdImTyxvZ8CXicKGX0pHX/5XeTH7bW14\nQWBvUO9NkLnY10aRLhm7tpiwRe6/YkWU1nPEzed0S//YOTuPrCnSlnJ2i9tu+6VF5vwYuTvNooAd\nct8qJ4dondTTtyySfc6CbOrJ/LFR3L+Rlu7EBO9Q/Q0TvQDKtLGGn8+i3izeeaJP+paSz4fwjzLn\n0IS0xVicSIs80/NOkQ3D31f2DAWOlHOwN99bl34IAHeVC70yiFIRKWsvB6rMYq1g8dLdHTLQKH6+\nkfghXF8pey9qjb9P3Ho5n6mTQ9Uy8B8oUURnyyKlXW+Uen990abEs7h5z/kAPPNLCWCvv3Ml8WCC\nMA4LXkYcgK6Uygf+DHxSa92Z/DctW94Pe3dKqRuVUiuUUitijPFmnmNMXMdZw/PM40RC6sDOwlSa\nSW3jIe0bMrMd8rdJYR9A3IuyOv4sc92TMtfGDK+nkPk2HhNt8Vi3MQPqKRwbNo4FI5oZKKXCyEDq\nd1rre83hPUqpSq11s1KqEtg73Ge11rcCtwIUqtIRDQcTy7DzjaQ8v4pdl8qs4bgckem2xWSmtKxj\nJgWbxIxpD5qd2weixKNHtuTR1z5reJ5pTGeqEuk0i2wGdB/ZKjcI7h4zG4e/CbOf3U9FkflRyfkA\nRKMhCjaaWeUCeQ6fariawmw575V1ki5h/r0ihw43wzqcfQO6L2gYY2JfoLK5pRL8SDSWUFgGg10H\ng1+DhIY9C2SmkdvYxb5TxQvwxlrZR+m6IpkV7/c1m2OSbK6vVWZIXVUh8lZ0sLplCZVuHRXuDLSO\nkaUPtJGkjdJHa+NBjHA/vSD9w9abcymImDQJ94giV/6oSTfQ0zesIjWR9VR7XsKt/EpMlItTnW4K\nHGlvt5z6RwB6jXp1Vk4LcKAydWtHFX95/SK53r51wY0c8v+mqi1qsxdmxdIOQiYVyUsm1cO2IqnH\n5+R0EMuW2f1bT5A921yluLdL1PHGZklu6bcFKQcmqC0OeabBoo6c5zaQ1SH1DfGSsO8EhVcj9+VH\npf0smi1BuztOLYanRa2a9pwoGu1zI0y9fhsAX6x4EoDspBCD/Z68KAc8k5S2t19s1M8d0kbGsy0m\nY9qlPlPSACw562f45vl+dMn1ANRveOmILzuh9dT3KH1agsb/eq54Lr5w6ROUutL2giztNWWm3+0t\nAJO4te0EqbvnXvUS5+aI629LXHSUJbcZ13Lb6mHbZTq8F4OdNXhZ3OS5m8WuyPQqotNEpcpplj7p\nm3+7Ar9Czq/7rdg49XFRko9WeRwph1WmlEwhbgPWa62/l/Sn+4F/Mz//G7Bk7G9vYtBa8woryKOA\nGWpu4ng5VTQjL7cYUZikNo7Evma2E+Jg6X6yoLXm5dZHyQ+XMsNdkDg+1EagPTV3OHoyvZ5C5tt4\nrLTFV/QK8ig8pI3YtpjWHAs2jiUjUabOBt4DrFVKrTLH/gP4FnCPUur9wHbgqjG5I+2Db5b6TxVl\nYutlWbhZMtr8o0kt//hGKdw5341S9YoEJntBssZ47Ih8oh20spsd5FPEUi0xWfUcxwzmsZal7NLb\n8IiB2Dzu6BdlBl9zrcQ6qKwsMIGiNWa20XlyJdtnyUxwarPxde+RWcfQmKSR2JdLhGyTFHNMMCqb\nZ7Y3CGKigMHtSIKYuFkzaDtZlMa8XVLO7ccV02bGRPkhs8VMTGbDxU4/W6Jm/8btJoB2ZQNNvevJ\nD5fxfGybsXHRQTZiFlOMKSa9gXJdHKPE6SBeIQisL8hn70WSNPHOL8mcZG44i61xUabesvqzAEy5\n32zpMEyy1VTU07r/k9ifG8pvAOArF/2Js3O3AbDdlEGRK7FdjfEQ+82WJT/c+wYANp8fwu9pYaSk\nsi0GM2C1ZiNT94pC+sppooBuqZbfj89qI2LKe0tM2ufXt7+Fvv8xgcDPySID7zVmwRPWFk3/53d3\n46yVBJ2l283Sf3cmrSJm8+Wz7wdgYbakA/CnO/xz9nwAwtdLHbysYA0VrkkyahayBDFRcQbr6f6X\n5BnR+U92s/1AG9XxzFDzWaufH9+2OAxuqbxHzvn5UgBmhlxWREVynftLicfxjjBRZSrqabxRymjh\nt6X+ff3UC/l8haSN6TELWGJGHWw7rZD248TD8/OLbwPgxOz2RCqL9798HQBVd4na4/UcHJCdNu/F\n4F1uFoEk9lzcsIXQFqmXs9abfrcgDx1sMRYspBhnRSpA6QnMPFyoSvXp6sLDnhdkeO1/g8iyXbUh\n+svkpVS82ewp9JhI7MHLerxZph+jU+9/bUe/YaQ2HjFBJmrzbJyKctrOkA68cJOs3HN3yWAqvvvo\ngu9HYuNR25e8N+CQTWRDtVXsukzcKTrIPKNBG920b5p8rnihdHxXTF/Nna+cDkDVb+R55K1uSgQq\n+ibnyHA8qv/0otb61EPd6pHaGKxAdMpKaX1TkOVbbCxolA5g7ykO77vsUQA+WTq4UmvZgAwGv/6v\n15kD4so82gDJMa+npqxC1TKo6D2uCmUGw7E813w3bhRHkbtPOq6cRyQ/03hkUx63tphYUZqkCp0g\nk7aGj8nL90On/JM/7xA3S/8/ZPBQefvqMd9w+6jbYtC2hqs/waA/HMKprwOg8SJx6dVdJnn5zi3b\nSKkZHIeVlOUbItvoMi/rcndwX7SAW3a/CYAN3xd3btFf1+AHe8Udoh6PR1tMxonIYHf7pyS/1pMf\nlJWmntZccMfnAZjxVbPZ+DjsRzde9VSZ7Pbe6QvZ9mE5Fu+WOuvmmQU/Gt51nGQ5f2OBTM6LnT4+\n3nANAPmfkvP9Bin3ox1wpPS9mLRwSblSH3U8fvBEdJRjnJHaaDOgWywWi8VisYyCtNybL8hvkv2g\njKxzIxGUcW8F+/YNl6Mmowlke7PkWbV1UPKsqHTBzuF+z+GzvaeM5NnBENlWd/dQvFnsikVkfB/u\n8YlsliBeZdIrdC8UJeDx7rMpniGKVGSDSLrxpt3jMrscCcGsztvbQsk9kjYgkJiDmVPdEo8n7hMX\n9a3XygxNl0WZ+0Oj3KxYM5G3PHJMWQUuhqzGwezQh4rqSbd9zUZEkFYgWU0zLvc515ul9PnVFA/s\nNOdJ0Lafono3LIeahZv71AMe3iui7Fc1SCnGfydukscrT6f5XPk5KjH4fK1YU2YE03YTOlO6zvRH\nYUVOm1y38MH02dNOZWez+UuS1uKOqyQDf6AcbIxHqHo2yAM39pmwx5ugfjrPrWXOBkkNo/JFhcO8\nH/oWVfFE8dkAPIF8z2vsI2+1uO29MVZSU4LWiYUWOpb6WmeVKYvFYrFYLJZRkH7KVHJsTaDG9PTA\nMAFyxyRBYHdXF5gd5xOpBiYo0G6s8Tu7yX3MJAM0mZh1V9eg+hjsX2j2uVOuS9mLZnf0/jRSKLUe\nzBodHEpSOYKFBfUvDcaM6ck8MzwWGKKiep2dhzh5EjFEhQt2JWDPXipWvdaHoDj4YUjco1zSP+Da\nqcSJRPjhlbcDcHaO3KOnJQ2Eh0PuRrOvYhrc61Hje3gtZnHHkDUe4V1NwyrHk0+HOwxpVH5WmbJY\nLBaLxWIZBemnTKXRSDOtSXpOk1WRCtCxKGZ7LzjESrxA9dFKTW41bojaYbFMOtK1DgeKWSjEfftP\nBmBhlizr3xKTILDrH/oA85vTNEbRMmlJv8GUxXI4tJ6cgyiLxTK+mEGe19LCtjNkwnWjKznPMBuL\nz429mF6LBiwZgXXzWSwWi8VisYyCCU3aqZRqAXqAfRP2T4+eKRx4nzO01uWH+5BSqgvYMG53NbYc\nsY2TvAwh820caT09Fmy0bTF9sG3xNThGbMzotggTPJgCUEqtOFzW23TgaO9zstgHmW/jaO7T2pg+\nZHo9hcy30dbT8fvsRJLp9RSO/l6tm89isVgsFotlFNjBlMVisVgsFssoSMVg6tYU/M+j4Wjvc7LY\nB5lv42ju09qYPmR6PYXMt9HW0/H77ESS6fUUjvJeJzxmymKxWCwWiyWTsG4+i8VisVgsllEwYYMp\npdTFSqkNSqlNSqmbJur/Hg6lVK1S6gml1CtKqXVKqU+Y419RSu1SSq0yX5eM4FrWxhQxVjamq32Q\n+TbaemptHHKdjLbPfMbamCLG0kYAtNbj/gW4wGZgFpAFrAYWTsT/HsG9VQInm58LgAZgIfAV4LPW\nxmPHxnS271iw0dZTa+OxYp+1MXNsDL4mSpl6HbBJa71Fax0F7gYum6D/fUi01s1a65fMz13AeqD6\nKC5lbUwhY2Rj2toHmW+jradHRKbbmOn2gbUxpYyhjcDEufmqgZ1JvzcyipseL5RSdcBJwDJz6KNK\nqTVKqduVUiWH+bi1MU0YhY2Twj7IfBttPT3mbcx0+8DamDaM0kbABqAnUErlA38GPqm17gR+BswG\nTgSage+m8PbGBGujtXEykOn2gbWRDLAx0+0DayNHYONEDaZ2AbVJv9eYY2mBUiqMPMzfaa3vBdBa\n79Fae1prH/g/RK48FNbGFDMGNqa1fZD5Ntp6am00ZLp9YG1MOWNkIzBxg6nlwByl1EylVBZwDXD/\nBP3vQ6KUUsBtwHqt9feSjlcmnXYF8PJhLmVtTCFjZGPa2geZb6OtpwmsjZlvH1gbU8oY2igcacT6\n0X4BlyDR8puB/5yo/zuC+zoH0MAaYJX5ugT4DbDWHL8fqLQ2Zr6N6WrfsWCjrafWxmPJPmtj5tio\ntbYZ0C0Wi8VisVhGgw1At1gsFovFYhkFdjBlsVgsFovFMgrsYMpisVgsFotlFNjBlMVisVgsFsso\nsIMpi8VisVgsllFgB1MWi8VisVgso8AOpiwWi8VisVhGgR1MWSwWi8VisYyC/w9QOi9ocbmRbwAA\nAABJRU5ErkJggg==\n",
            "text/plain": [
              "\u003cFigure size 720x72 with 10 Axes\u003e"
            ]
          },
          "metadata": {
            "tags": []
          },
          "output_type": "display_data"
        }
      ],
      "source": [
        "num_images = 10\n",
        "noise = get_noise_batch(num_images)\n",
        "gen_images = gan.generate(noise)\n",
        "plt.rcParams['figure.figsize'] = (num_images, 1)\n",
        "for i in range(num_images):\n",
        "  plt.subplot(1, num_images, i+1)\n",
        "  plt.imshow(gen_images[i])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "Lnkc55PtqA_I"
      },
      "source": [
        "Whew, those should already look much closer to actual handwritten digits! But there's still a ways to go. There are many potential ways to improve the results further, including (roughly in order of implementation difficulty):\n",
        "\n",
        "* Train the model for more steps (e.g., try setting `num_epochs = 100` or higher).\n",
        "* Add learning rate decay to make the step size smaller after a certain amount of time. (For example, try `optimizer = GANOptimizer(num_epochs=100, decay_lr_start_epoch=50)` to begin decaying the learning rate halfway through training.)\n",
        "* Increase the size of the model -- make it deeper (`num_layers=4`) and/or wider (`embed_dim=2048`).\n",
        "* Tweak the learning hyperparameters, such as the discriminator & generator learning rates or the training batch size.\n",
        "* Make the models **convolutional**, using `snt.Conv2D` layers to process the inputs and outputs. The generator and discriminator defined above use MLPs, which naively treat the digit images as flat vectors.\n",
        "* Make the model **conditional** on labels by feeding the digit label (e.g., \"7\") into the generator and discriminator. (See [1, 2] for some ideas on how to do this.)\n",
        "\n",
        "[1] M. Mirza and S. Osindero. [*Conditional Generative Adversarial Networks*](https://arxiv.org/abs/1411.1784). *arXiv:1411.1784*, 2014.\n",
        "\n",
        "[2] T. Miyato and M. Koyama. [*cGANs with Projection Discriminator*](https://arxiv.org/abs/1802.05637). *arXiv:1802.05637*, 2018."
      ]
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "collapsed_sections": [],
      "name": "little_gan_on_mnist.ipynb",
      "provenance": [],
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}
